2023-02-09 22:53:55,722 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-09 22:53:55,951 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a 2023-02-09 22:53:55,965 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 22:53:55,965 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d, deleteOnExit=true 2023-02-09 22:53:55,966 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 22:53:55,966 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/test.cache.data in system properties and HBase conf 2023-02-09 22:53:55,966 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 22:53:55,967 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/hadoop.log.dir in system properties and HBase conf 2023-02-09 22:53:55,967 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 22:53:55,968 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 22:53:55,968 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 22:53:56,078 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 22:53:56,447 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 22:53:56,450 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 22:53:56,451 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 22:53:56,451 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 22:53:56,451 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 22:53:56,452 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 22:53:56,452 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 22:53:56,452 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 22:53:56,452 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 22:53:56,453 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 22:53:56,453 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/nfs.dump.dir in system properties and HBase conf 2023-02-09 22:53:56,453 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/java.io.tmpdir in system properties and HBase conf 2023-02-09 22:53:56,453 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 22:53:56,454 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 22:53:56,454 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 22:53:56,943 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 22:53:56,947 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 22:53:57,236 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 22:53:57,405 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 22:53:57,419 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 22:53:57,455 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 22:53:57,486 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/java.io.tmpdir/Jetty_localhost_46717_hdfs____.805l6j/webapp 2023-02-09 22:53:57,637 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46717 2023-02-09 22:53:57,648 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 22:53:57,648 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 22:53:58,072 WARN [Listener at localhost/41659] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 22:53:58,181 WARN [Listener at localhost/41659] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 22:53:58,205 WARN [Listener at localhost/41659] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 22:53:58,212 INFO [Listener at localhost/41659] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 22:53:58,216 INFO [Listener at localhost/41659] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/java.io.tmpdir/Jetty_localhost_34619_datanode____.ghgkos/webapp 2023-02-09 22:53:58,315 INFO [Listener at localhost/41659] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34619 2023-02-09 22:53:58,629 WARN [Listener at localhost/44083] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 22:53:58,664 WARN [Listener at localhost/44083] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 22:53:58,668 WARN [Listener at localhost/44083] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 22:53:58,670 INFO [Listener at localhost/44083] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 22:53:58,678 INFO [Listener at localhost/44083] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/java.io.tmpdir/Jetty_localhost_41693_datanode____.6jq15y/webapp 2023-02-09 22:53:58,791 INFO [Listener at localhost/44083] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41693 2023-02-09 22:53:58,805 WARN [Listener at localhost/36851] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 22:53:58,822 WARN [Listener at localhost/36851] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 22:53:58,825 WARN [Listener at localhost/36851] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 22:53:58,827 INFO [Listener at localhost/36851] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 22:53:58,832 INFO [Listener at localhost/36851] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/java.io.tmpdir/Jetty_localhost_37071_datanode____ac5j4d/webapp 2023-02-09 22:53:58,938 INFO [Listener at localhost/36851] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37071 2023-02-09 22:53:58,948 WARN [Listener at localhost/44743] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 22:53:59,145 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6920f935b54bab5b: Processing first storage report for DS-3449ba3c-78c9-434e-b433-75072f6d0618 from datanode 4256d769-ee79-40f3-a328-ac20eb1446de 2023-02-09 22:53:59,147 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6920f935b54bab5b: from storage DS-3449ba3c-78c9-434e-b433-75072f6d0618 node DatanodeRegistration(127.0.0.1:44159, datanodeUuid=4256d769-ee79-40f3-a328-ac20eb1446de, infoPort=41825, infoSecurePort=0, ipcPort=36851, storageInfo=lv=-57;cid=testClusterID;nsid=411914157;c=1675983237018), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-09 22:53:59,147 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x76d4f2c621df94c6: Processing first storage report for DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70 from datanode 73c0fc0c-d116-48f6-bf3b-b31855654d02 2023-02-09 22:53:59,148 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x76d4f2c621df94c6: from storage DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70 node DatanodeRegistration(127.0.0.1:34043, datanodeUuid=73c0fc0c-d116-48f6-bf3b-b31855654d02, infoPort=35401, infoSecurePort=0, ipcPort=44083, storageInfo=lv=-57;cid=testClusterID;nsid=411914157;c=1675983237018), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 22:53:59,148 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb94493a67d559cbc: Processing first storage report for DS-e2fd4acd-65ed-445e-9924-96b6b5507938 from datanode b11f84d6-5ece-443b-9b52-3e4a014154a8 2023-02-09 22:53:59,148 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb94493a67d559cbc: from storage DS-e2fd4acd-65ed-445e-9924-96b6b5507938 node DatanodeRegistration(127.0.0.1:38327, datanodeUuid=b11f84d6-5ece-443b-9b52-3e4a014154a8, infoPort=39971, infoSecurePort=0, ipcPort=44743, storageInfo=lv=-57;cid=testClusterID;nsid=411914157;c=1675983237018), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 22:53:59,148 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6920f935b54bab5b: Processing first storage report for DS-7576c167-2044-418a-8e91-d1399f6e5b75 from datanode 4256d769-ee79-40f3-a328-ac20eb1446de 2023-02-09 22:53:59,148 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6920f935b54bab5b: from storage DS-7576c167-2044-418a-8e91-d1399f6e5b75 node DatanodeRegistration(127.0.0.1:44159, datanodeUuid=4256d769-ee79-40f3-a328-ac20eb1446de, infoPort=41825, infoSecurePort=0, ipcPort=36851, storageInfo=lv=-57;cid=testClusterID;nsid=411914157;c=1675983237018), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 22:53:59,149 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x76d4f2c621df94c6: Processing first storage report for DS-47458d37-faa6-4043-90ac-b0d8c547d0eb from datanode 73c0fc0c-d116-48f6-bf3b-b31855654d02 2023-02-09 22:53:59,149 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x76d4f2c621df94c6: from storage DS-47458d37-faa6-4043-90ac-b0d8c547d0eb node DatanodeRegistration(127.0.0.1:34043, datanodeUuid=73c0fc0c-d116-48f6-bf3b-b31855654d02, infoPort=35401, infoSecurePort=0, ipcPort=44083, storageInfo=lv=-57;cid=testClusterID;nsid=411914157;c=1675983237018), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 22:53:59,149 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb94493a67d559cbc: Processing first storage report for DS-18320dfc-d3de-4088-8f82-edcd04a3ca50 from datanode b11f84d6-5ece-443b-9b52-3e4a014154a8 2023-02-09 22:53:59,149 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb94493a67d559cbc: from storage DS-18320dfc-d3de-4088-8f82-edcd04a3ca50 node DatanodeRegistration(127.0.0.1:38327, datanodeUuid=b11f84d6-5ece-443b-9b52-3e4a014154a8, infoPort=39971, infoSecurePort=0, ipcPort=44743, storageInfo=lv=-57;cid=testClusterID;nsid=411914157;c=1675983237018), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 22:53:59,336 DEBUG [Listener at localhost/44743] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a 2023-02-09 22:53:59,399 INFO [Listener at localhost/44743] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/zookeeper_0, clientPort=49544, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 22:53:59,413 INFO [Listener at localhost/44743] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=49544 2023-02-09 22:53:59,424 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:59,427 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:00,115 INFO [Listener at localhost/44743] util.FSUtils(479): Created version file at hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976 with version=8 2023-02-09 22:54:00,115 INFO [Listener at localhost/44743] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/hbase-staging 2023-02-09 22:54:00,419 INFO [Listener at localhost/44743] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 22:54:00,905 INFO [Listener at localhost/44743] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=45 2023-02-09 22:54:00,937 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:00,937 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:00,938 INFO [Listener at localhost/44743] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 22:54:00,938 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:00,938 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 22:54:01,080 INFO [Listener at localhost/44743] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 22:54:01,150 DEBUG [Listener at localhost/44743] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 22:54:01,243 INFO [Listener at localhost/44743] ipc.NettyRpcServer(120): Bind to /172.31.2.10:44215 2023-02-09 22:54:01,253 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,255 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,275 INFO [Listener at localhost/44743] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44215 connecting to ZooKeeper ensemble=127.0.0.1:49544 2023-02-09 22:54:01,315 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:442150x0, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 22:54:01,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:44215-0x1003b4b98c70000 connected 2023-02-09 22:54:01,340 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:54:01,341 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:54:01,344 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 22:54:01,352 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44215 2023-02-09 22:54:01,352 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44215 2023-02-09 22:54:01,352 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44215 2023-02-09 22:54:01,353 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44215 2023-02-09 22:54:01,353 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44215 2023-02-09 22:54:01,359 INFO [Listener at localhost/44743] master.HMaster(439): hbase.rootdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976, hbase.cluster.distributed=false 2023-02-09 22:54:01,428 INFO [Listener at localhost/44743] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=45 2023-02-09 22:54:01,429 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,429 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,429 INFO [Listener at localhost/44743] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 22:54:01,429 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,429 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 22:54:01,434 INFO [Listener at localhost/44743] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 22:54:01,438 INFO [Listener at localhost/44743] ipc.NettyRpcServer(120): Bind to /172.31.2.10:41885 2023-02-09 22:54:01,440 INFO [Listener at localhost/44743] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 22:54:01,446 DEBUG [Listener at localhost/44743] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 22:54:01,448 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,449 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,451 INFO [Listener at localhost/44743] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41885 connecting to ZooKeeper ensemble=127.0.0.1:49544 2023-02-09 22:54:01,455 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:418850x0, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 22:54:01,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41885-0x1003b4b98c70001 connected 2023-02-09 22:54:01,457 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:54:01,458 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:54:01,459 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 22:54:01,459 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41885 2023-02-09 22:54:01,460 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41885 2023-02-09 22:54:01,461 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41885 2023-02-09 22:54:01,461 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41885 2023-02-09 22:54:01,461 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41885 2023-02-09 22:54:01,474 INFO [Listener at localhost/44743] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=45 2023-02-09 22:54:01,474 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,474 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,475 INFO [Listener at localhost/44743] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 22:54:01,475 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,475 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 22:54:01,475 INFO [Listener at localhost/44743] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 22:54:01,477 INFO [Listener at localhost/44743] ipc.NettyRpcServer(120): Bind to /172.31.2.10:44627 2023-02-09 22:54:01,477 INFO [Listener at localhost/44743] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 22:54:01,479 DEBUG [Listener at localhost/44743] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 22:54:01,480 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,483 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,485 INFO [Listener at localhost/44743] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44627 connecting to ZooKeeper ensemble=127.0.0.1:49544 2023-02-09 22:54:01,489 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:446270x0, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 22:54:01,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:44627-0x1003b4b98c70002 connected 2023-02-09 22:54:01,490 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:54:01,491 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:54:01,492 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 22:54:01,492 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44627 2023-02-09 22:54:01,492 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44627 2023-02-09 22:54:01,493 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44627 2023-02-09 22:54:01,493 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44627 2023-02-09 22:54:01,493 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44627 2023-02-09 22:54:01,506 INFO [Listener at localhost/44743] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=45 2023-02-09 22:54:01,506 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,506 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,506 INFO [Listener at localhost/44743] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 22:54:01,506 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:54:01,506 INFO [Listener at localhost/44743] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 22:54:01,507 INFO [Listener at localhost/44743] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 22:54:01,508 INFO [Listener at localhost/44743] ipc.NettyRpcServer(120): Bind to /172.31.2.10:40413 2023-02-09 22:54:01,508 INFO [Listener at localhost/44743] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 22:54:01,509 DEBUG [Listener at localhost/44743] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 22:54:01,510 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,512 INFO [Listener at localhost/44743] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,513 INFO [Listener at localhost/44743] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40413 connecting to ZooKeeper ensemble=127.0.0.1:49544 2023-02-09 22:54:01,517 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:404130x0, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 22:54:01,518 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40413-0x1003b4b98c70003 connected 2023-02-09 22:54:01,518 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:54:01,519 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:54:01,520 DEBUG [Listener at localhost/44743] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 22:54:01,521 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40413 2023-02-09 22:54:01,521 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40413 2023-02-09 22:54:01,521 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40413 2023-02-09 22:54:01,521 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40413 2023-02-09 22:54:01,522 DEBUG [Listener at localhost/44743] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40413 2023-02-09 22:54:01,523 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:01,532 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 22:54:01,533 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:01,551 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 22:54:01,551 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 22:54:01,551 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 22:54:01,551 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 22:54:01,553 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:01,553 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 22:54:01,554 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase9.apache.org,44215,1675983240249 from backup master directory 2023-02-09 22:54:01,555 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 22:54:01,558 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:01,558 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 22:54:01,559 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 22:54:01,559 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:01,561 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 22:54:01,563 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 22:54:01,652 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/hbase.id with ID: 540c617a-10c9-446f-aa0b-d8659f011eb6 2023-02-09 22:54:01,693 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:54:01,709 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:01,755 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x452b56e4 to 127.0.0.1:49544 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:01,788 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a563eec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:01,814 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 22:54:01,816 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 22:54:01,832 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 22:54:01,833 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 22:54:01,835 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:54:01,840 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:54:01,842 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:54:01,875 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store-tmp 2023-02-09 22:54:01,910 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:01,911 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 22:54:01,911 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:54:01,911 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:54:01,911 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 22:54:01,911 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:54:01,911 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:54:01,911 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675983241910Disabling compacts and flushes for region at 1675983241911 (+1 ms)Disabling writes for close at 1675983241911Writing region close event to WAL at 1675983241911Closed at 1675983241911 2023-02-09 22:54:01,913 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/WALs/jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:01,935 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C44215%2C1675983240249, suffix=, logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/WALs/jenkins-hbase9.apache.org,44215,1675983240249, archiveDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/oldWALs, maxLogs=10 2023-02-09 22:54:01,999 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK] 2023-02-09 22:54:01,999 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK] 2023-02-09 22:54:01,999 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK] 2023-02-09 22:54:02,008 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:54:02,081 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/WALs/jenkins-hbase9.apache.org,44215,1675983240249/jenkins-hbase9.apache.org%2C44215%2C1675983240249.1675983241946 2023-02-09 22:54:02,081 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK], DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK], DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK]] 2023-02-09 22:54:02,082 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:02,082 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:02,086 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:54:02,087 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:54:02,146 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:54:02,154 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 22:54:02,179 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 22:54:02,191 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:02,198 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:54:02,201 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:54:02,214 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:54:02,218 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:02,219 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72718899, jitterRate=0.08359603583812714}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:02,219 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675983242115Initializing all the Stores at 1675983242117 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983242118 (+1 ms)Cleaning up temporary data from old regions at 1675983242207 (+89 ms)Cleaning up detritus from prior splits at 1675983242208 (+1 ms)Region opened successfully at 1675983242219 (+11 ms) 2023-02-09 22:54:02,220 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 22:54:02,240 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:02,241 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:02,243 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:02,245 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 22:54:02,277 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-02-09 22:54:02,277 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 22:54:02,302 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 22:54:02,307 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 22:54:02,330 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 22:54:02,334 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 22:54:02,335 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 22:54:02,340 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 22:54:02,344 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 22:54:02,346 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:02,348 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 22:54:02,349 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 22:54:02,362 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 22:54:02,367 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 22:54:02,367 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 22:54:02,367 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 22:54:02,367 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 22:54:02,367 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:02,368 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase9.apache.org,44215,1675983240249, sessionid=0x1003b4b98c70000, setting cluster-up flag (Was=false) 2023-02-09 22:54:02,387 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:02,394 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 22:54:02,396 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:02,401 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:02,407 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 22:54:02,408 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:02,411 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.hbase-snapshot/.tmp 2023-02-09 22:54:02,425 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(952): ClusterId : 540c617a-10c9-446f-aa0b-d8659f011eb6 2023-02-09 22:54:02,425 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(952): ClusterId : 540c617a-10c9-446f-aa0b-d8659f011eb6 2023-02-09 22:54:02,425 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(952): ClusterId : 540c617a-10c9-446f-aa0b-d8659f011eb6 2023-02-09 22:54:02,431 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 22:54:02,431 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 22:54:02,431 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 22:54:02,437 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 22:54:02,437 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 22:54:02,437 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 22:54:02,438 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 22:54:02,437 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 22:54:02,438 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 22:54:02,443 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 22:54:02,443 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 22:54:02,443 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 22:54:02,445 DEBUG [RS:0;jenkins-hbase9:41885] zookeeper.ReadOnlyZKClient(139): Connect 0x3d2bef64 to 127.0.0.1:49544 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:02,446 DEBUG [RS:2;jenkins-hbase9:40413] zookeeper.ReadOnlyZKClient(139): Connect 0x736d636d to 127.0.0.1:49544 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:02,446 DEBUG [RS:1;jenkins-hbase9:44627] zookeeper.ReadOnlyZKClient(139): Connect 0x7e4776d0 to 127.0.0.1:49544 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:02,478 DEBUG [RS:0;jenkins-hbase9:41885] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33ec16b1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:02,478 DEBUG [RS:1;jenkins-hbase9:44627] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70809768, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:02,479 DEBUG [RS:2;jenkins-hbase9:40413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7daa9370, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:02,479 DEBUG [RS:0;jenkins-hbase9:41885] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37d38414, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-02-09 22:54:02,479 DEBUG [RS:1;jenkins-hbase9:44627] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74bf1da0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-02-09 22:54:02,479 DEBUG [RS:2;jenkins-hbase9:40413] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7995081d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-02-09 22:54:02,504 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase9:40413 2023-02-09 22:54:02,507 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase9:44627 2023-02-09 22:54:02,507 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:41885 2023-02-09 22:54:02,509 INFO [RS:0;jenkins-hbase9:41885] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 22:54:02,509 INFO [RS:1;jenkins-hbase9:44627] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 22:54:02,509 INFO [RS:2;jenkins-hbase9:40413] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 22:54:02,510 INFO [RS:1;jenkins-hbase9:44627] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 22:54:02,510 INFO [RS:0;jenkins-hbase9:41885] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 22:54:02,510 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 22:54:02,510 INFO [RS:2;jenkins-hbase9:40413] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 22:54:02,511 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 22:54:02,510 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 22:54:02,513 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,44215,1675983240249 with isa=jenkins-hbase9.apache.org/172.31.2.10:41885, startcode=1675983241428 2023-02-09 22:54:02,513 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,44215,1675983240249 with isa=jenkins-hbase9.apache.org/172.31.2.10:44627, startcode=1675983241473 2023-02-09 22:54:02,513 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,44215,1675983240249 with isa=jenkins-hbase9.apache.org/172.31.2.10:40413, startcode=1675983241505 2023-02-09 22:54:02,535 DEBUG [RS:2;jenkins-hbase9:40413] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 22:54:02,535 DEBUG [RS:1;jenkins-hbase9:44627] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 22:54:02,535 DEBUG [RS:0;jenkins-hbase9:41885] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 22:54:02,584 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:02,593 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:54:02,593 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:54:02,594 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:54:02,594 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:54:02,594 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-02-09 22:54:02,594 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,594 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-02-09 22:54:02,594 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,594 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:36611, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 22:54:02,594 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50059, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 22:54:02,594 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50335, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 22:54:02,595 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983272595 2023-02-09 22:54:02,597 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 22:54:02,607 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:02,607 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 22:54:02,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 22:54:02,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44215] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 22:54:02,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44215] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 22:54:02,612 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 22:54:02,614 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 22:54:02,622 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 22:54:02,623 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 22:54:02,623 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 22:54:02,623 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 22:54:02,625 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,627 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 22:54:02,628 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 22:54:02,629 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 22:54:02,631 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 22:54:02,632 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 22:54:02,634 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1675983242633,5,FailOnTimeoutGroup] 2023-02-09 22:54:02,634 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1675983242634,5,FailOnTimeoutGroup] 2023-02-09 22:54:02,634 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,634 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 22:54:02,636 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,636 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,645 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 22:54:02,645 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 22:54:02,645 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 22:54:02,645 WARN [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 22:54:02,646 WARN [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 22:54:02,646 WARN [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 22:54:02,660 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:02,661 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:02,661 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976 2023-02-09 22:54:02,683 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:02,685 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 22:54:02,687 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/info 2023-02-09 22:54:02,688 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 22:54:02,689 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:02,689 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 22:54:02,691 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/rep_barrier 2023-02-09 22:54:02,692 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 22:54:02,693 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:02,693 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 22:54:02,695 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/table 2023-02-09 22:54:02,695 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 22:54:02,696 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:02,697 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740 2023-02-09 22:54:02,698 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740 2023-02-09 22:54:02,701 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 22:54:02,703 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 22:54:02,707 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:02,707 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64919727, jitterRate=-0.03262068331241608}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 22:54:02,707 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675983242683Initializing all the Stores at 1675983242684 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983242685 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983242685Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983242685Cleaning up temporary data from old regions at 1675983242700 (+15 ms)Cleaning up detritus from prior splits at 1675983242700Region opened successfully at 1675983242707 (+7 ms) 2023-02-09 22:54:02,708 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 22:54:02,708 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 22:54:02,708 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 22:54:02,708 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 22:54:02,708 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 22:54:02,709 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 22:54:02,709 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675983242708Disabling compacts and flushes for region at 1675983242708Disabling writes for close at 1675983242708Writing region close event to WAL at 1675983242709 (+1 ms)Closed at 1675983242709 2023-02-09 22:54:02,715 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:02,715 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 22:54:02,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 22:54:02,745 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 22:54:02,747 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 22:54:02,747 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,44215,1675983240249 with isa=jenkins-hbase9.apache.org/172.31.2.10:41885, startcode=1675983241428 2023-02-09 22:54:02,747 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,44215,1675983240249 with isa=jenkins-hbase9.apache.org/172.31.2.10:40413, startcode=1675983241505 2023-02-09 22:54:02,748 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,44215,1675983240249 with isa=jenkins-hbase9.apache.org/172.31.2.10:44627, startcode=1675983241473 2023-02-09 22:54:02,755 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,755 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44215] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44215] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,762 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976 2023-02-09 22:54:02,762 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976 2023-02-09 22:54:02,762 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976 2023-02-09 22:54:02,762 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41659 2023-02-09 22:54:02,762 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41659 2023-02-09 22:54:02,762 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 22:54:02,762 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41659 2023-02-09 22:54:02,762 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 22:54:02,762 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 22:54:02,776 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:54:02,777 DEBUG [RS:1;jenkins-hbase9:44627] zookeeper.ZKUtil(162): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,777 DEBUG [RS:2;jenkins-hbase9:40413] zookeeper.ZKUtil(162): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,777 WARN [RS:1;jenkins-hbase9:44627] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 22:54:02,777 WARN [RS:2;jenkins-hbase9:40413] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 22:54:02,778 INFO [RS:1;jenkins-hbase9:44627] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:54:02,778 INFO [RS:2;jenkins-hbase9:40413] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:54:02,778 DEBUG [RS:0;jenkins-hbase9:41885] zookeeper.ZKUtil(162): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,778 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,779 WARN [RS:0;jenkins-hbase9:41885] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 22:54:02,779 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,779 INFO [RS:0;jenkins-hbase9:41885] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:54:02,779 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,40413,1675983241505] 2023-02-09 22:54:02,779 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,779 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,41885,1675983241428] 2023-02-09 22:54:02,779 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,44627,1675983241473] 2023-02-09 22:54:02,791 DEBUG [RS:0;jenkins-hbase9:41885] zookeeper.ZKUtil(162): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,791 DEBUG [RS:2;jenkins-hbase9:40413] zookeeper.ZKUtil(162): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,791 DEBUG [RS:1;jenkins-hbase9:44627] zookeeper.ZKUtil(162): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,792 DEBUG [RS:0;jenkins-hbase9:41885] zookeeper.ZKUtil(162): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,792 DEBUG [RS:2;jenkins-hbase9:40413] zookeeper.ZKUtil(162): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,792 DEBUG [RS:1;jenkins-hbase9:44627] zookeeper.ZKUtil(162): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,792 DEBUG [RS:0;jenkins-hbase9:41885] zookeeper.ZKUtil(162): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,792 DEBUG [RS:2;jenkins-hbase9:40413] zookeeper.ZKUtil(162): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,792 DEBUG [RS:1;jenkins-hbase9:44627] zookeeper.ZKUtil(162): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,802 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 22:54:02,802 DEBUG [RS:2;jenkins-hbase9:40413] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 22:54:02,802 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 22:54:02,811 INFO [RS:1;jenkins-hbase9:44627] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 22:54:02,811 INFO [RS:2;jenkins-hbase9:40413] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 22:54:02,811 INFO [RS:0;jenkins-hbase9:41885] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 22:54:02,832 INFO [RS:0;jenkins-hbase9:41885] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 22:54:02,832 INFO [RS:2;jenkins-hbase9:40413] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 22:54:02,832 INFO [RS:1;jenkins-hbase9:44627] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 22:54:02,836 INFO [RS:1;jenkins-hbase9:44627] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:54:02,836 INFO [RS:2;jenkins-hbase9:40413] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:54:02,837 INFO [RS:1;jenkins-hbase9:44627] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,836 INFO [RS:0;jenkins-hbase9:41885] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:54:02,837 INFO [RS:2;jenkins-hbase9:40413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,837 INFO [RS:0;jenkins-hbase9:41885] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,838 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 22:54:02,838 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 22:54:02,839 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 22:54:02,846 INFO [RS:0;jenkins-hbase9:41885] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,846 INFO [RS:2;jenkins-hbase9:40413] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,846 INFO [RS:1;jenkins-hbase9:44627] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,846 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,846 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,846 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-02-09 22:54:02,847 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,846 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:1;jenkins-hbase9:44627] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,847 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,848 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,848 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,848 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-02-09 22:54:02,848 DEBUG [RS:0;jenkins-hbase9:41885] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,848 INFO [RS:1;jenkins-hbase9:44627] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,848 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,848 INFO [RS:1;jenkins-hbase9:44627] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,849 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,849 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,849 DEBUG [RS:2;jenkins-hbase9:40413] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:54:02,851 INFO [RS:1;jenkins-hbase9:44627] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,851 INFO [RS:0;jenkins-hbase9:41885] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,851 INFO [RS:0;jenkins-hbase9:41885] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,851 INFO [RS:0;jenkins-hbase9:41885] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,853 INFO [RS:2;jenkins-hbase9:40413] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,853 INFO [RS:2;jenkins-hbase9:40413] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,853 INFO [RS:2;jenkins-hbase9:40413] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,870 INFO [RS:2;jenkins-hbase9:40413] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 22:54:02,870 INFO [RS:0;jenkins-hbase9:41885] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 22:54:02,870 INFO [RS:1;jenkins-hbase9:44627] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 22:54:02,874 INFO [RS:0;jenkins-hbase9:41885] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,41885,1675983241428-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,874 INFO [RS:1;jenkins-hbase9:44627] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44627,1675983241473-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,874 INFO [RS:2;jenkins-hbase9:40413] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,40413,1675983241505-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:02,899 INFO [RS:2;jenkins-hbase9:40413] regionserver.Replication(203): jenkins-hbase9.apache.org,40413,1675983241505 started 2023-02-09 22:54:02,900 INFO [RS:1;jenkins-hbase9:44627] regionserver.Replication(203): jenkins-hbase9.apache.org,44627,1675983241473 started 2023-02-09 22:54:02,900 INFO [RS:0;jenkins-hbase9:41885] regionserver.Replication(203): jenkins-hbase9.apache.org,41885,1675983241428 started 2023-02-09 22:54:02,900 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1638): Serving as jenkins-hbase9.apache.org,44627,1675983241473, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:44627, sessionid=0x1003b4b98c70002 2023-02-09 22:54:02,900 DEBUG [jenkins-hbase9:44215] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-09 22:54:02,900 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1638): Serving as jenkins-hbase9.apache.org,40413,1675983241505, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:40413, sessionid=0x1003b4b98c70003 2023-02-09 22:54:02,900 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1638): Serving as jenkins-hbase9.apache.org,41885,1675983241428, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:41885, sessionid=0x1003b4b98c70001 2023-02-09 22:54:02,900 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 22:54:02,900 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 22:54:02,900 DEBUG [RS:1;jenkins-hbase9:44627] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,900 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 22:54:02,900 DEBUG [RS:1;jenkins-hbase9:44627] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,44627,1675983241473' 2023-02-09 22:54:02,900 DEBUG [RS:2;jenkins-hbase9:40413] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,900 DEBUG [RS:0;jenkins-hbase9:41885] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,900 DEBUG [RS:2;jenkins-hbase9:40413] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,40413,1675983241505' 2023-02-09 22:54:02,901 DEBUG [RS:2;jenkins-hbase9:40413] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:02,900 DEBUG [RS:1;jenkins-hbase9:44627] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:02,901 DEBUG [RS:0;jenkins-hbase9:41885] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,41885,1675983241428' 2023-02-09 22:54:02,901 DEBUG [RS:0;jenkins-hbase9:41885] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:02,901 DEBUG [RS:2;jenkins-hbase9:40413] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:02,901 DEBUG [RS:1;jenkins-hbase9:44627] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:02,901 DEBUG [RS:0;jenkins-hbase9:41885] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:02,902 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 22:54:02,902 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 22:54:02,902 DEBUG [RS:2;jenkins-hbase9:40413] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:02,902 DEBUG [RS:2;jenkins-hbase9:40413] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,40413,1675983241505' 2023-02-09 22:54:02,902 DEBUG [RS:2;jenkins-hbase9:40413] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 22:54:02,903 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 22:54:02,903 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 22:54:02,903 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 22:54:02,903 DEBUG [RS:1;jenkins-hbase9:44627] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:02,903 DEBUG [RS:1;jenkins-hbase9:44627] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,44627,1675983241473' 2023-02-09 22:54:02,903 DEBUG [RS:1;jenkins-hbase9:44627] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 22:54:02,903 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 22:54:02,903 DEBUG [RS:2;jenkins-hbase9:40413] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 22:54:02,903 DEBUG [RS:0;jenkins-hbase9:41885] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:02,903 DEBUG [RS:0;jenkins-hbase9:41885] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,41885,1675983241428' 2023-02-09 22:54:02,903 DEBUG [RS:0;jenkins-hbase9:41885] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 22:54:02,903 DEBUG [RS:1;jenkins-hbase9:44627] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 22:54:02,904 DEBUG [RS:2;jenkins-hbase9:40413] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 22:54:02,904 INFO [RS:2;jenkins-hbase9:40413] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 22:54:02,904 DEBUG [RS:0;jenkins-hbase9:41885] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 22:54:02,904 INFO [RS:2;jenkins-hbase9:40413] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 22:54:02,904 DEBUG [RS:1;jenkins-hbase9:44627] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 22:54:02,904 INFO [RS:1;jenkins-hbase9:44627] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 22:54:02,904 INFO [RS:1;jenkins-hbase9:44627] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 22:54:02,904 DEBUG [RS:0;jenkins-hbase9:41885] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 22:54:02,904 INFO [RS:0;jenkins-hbase9:41885] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 22:54:02,904 INFO [RS:0;jenkins-hbase9:41885] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 22:54:02,905 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:02,911 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:02,912 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:02,912 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:02,912 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:02,914 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,44627,1675983241473, state=OPENING 2023-02-09 22:54:02,921 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 22:54:02,923 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:02,923 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 22:54:02,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:03,015 INFO [RS:2;jenkins-hbase9:40413] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C40413%2C1675983241505, suffix=, logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,40413,1675983241505, archiveDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs, maxLogs=32 2023-02-09 22:54:03,015 INFO [RS:0;jenkins-hbase9:41885] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C41885%2C1675983241428, suffix=, logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,41885,1675983241428, archiveDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs, maxLogs=32 2023-02-09 22:54:03,015 INFO [RS:1;jenkins-hbase9:44627] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C44627%2C1675983241473, suffix=, logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,44627,1675983241473, archiveDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs, maxLogs=32 2023-02-09 22:54:03,036 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK] 2023-02-09 22:54:03,036 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK] 2023-02-09 22:54:03,036 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK] 2023-02-09 22:54:03,045 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK] 2023-02-09 22:54:03,045 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK] 2023-02-09 22:54:03,045 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK] 2023-02-09 22:54:03,046 INFO [RS:2;jenkins-hbase9:40413] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,40413,1675983241505/jenkins-hbase9.apache.org%2C40413%2C1675983241505.1675983243018 2023-02-09 22:54:03,051 DEBUG [RS:2;jenkins-hbase9:40413] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK], DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK], DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK]] 2023-02-09 22:54:03,055 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK] 2023-02-09 22:54:03,056 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK] 2023-02-09 22:54:03,056 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK] 2023-02-09 22:54:03,061 INFO [RS:0;jenkins-hbase9:41885] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,41885,1675983241428/jenkins-hbase9.apache.org%2C41885%2C1675983241428.1675983243019 2023-02-09 22:54:03,062 DEBUG [RS:0;jenkins-hbase9:41885] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK], DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK], DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK]] 2023-02-09 22:54:03,062 INFO [RS:1;jenkins-hbase9:44627] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,44627,1675983241473/jenkins-hbase9.apache.org%2C44627%2C1675983241473.1675983243019 2023-02-09 22:54:03,066 DEBUG [RS:1;jenkins-hbase9:44627] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK], DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK], DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK]] 2023-02-09 22:54:03,109 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:03,111 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 22:54:03,115 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:44690, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 22:54:03,126 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 22:54:03,127 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:54:03,130 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C44627%2C1675983241473.meta, suffix=.meta, logDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,44627,1675983241473, archiveDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs, maxLogs=32 2023-02-09 22:54:03,149 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK] 2023-02-09 22:54:03,150 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK] 2023-02-09 22:54:03,154 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK] 2023-02-09 22:54:03,161 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/WALs/jenkins-hbase9.apache.org,44627,1675983241473/jenkins-hbase9.apache.org%2C44627%2C1675983241473.meta.1675983243131.meta 2023-02-09 22:54:03,162 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34043,DS-0c824c50-2c34-4b3a-a932-aa83d83a5c70,DISK], DatanodeInfoWithStorage[127.0.0.1:38327,DS-e2fd4acd-65ed-445e-9924-96b6b5507938,DISK], DatanodeInfoWithStorage[127.0.0.1:44159,DS-3449ba3c-78c9-434e-b433-75072f6d0618,DISK]] 2023-02-09 22:54:03,162 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:03,166 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:03,167 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 22:54:03,188 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 22:54:03,190 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 22:54:03,195 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 22:54:03,195 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:03,195 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 22:54:03,195 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 22:54:03,198 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 22:54:03,200 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/info 2023-02-09 22:54:03,200 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/info 2023-02-09 22:54:03,200 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 22:54:03,201 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:03,201 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 22:54:03,203 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/rep_barrier 2023-02-09 22:54:03,203 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/rep_barrier 2023-02-09 22:54:03,203 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 22:54:03,204 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:03,204 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 22:54:03,206 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/table 2023-02-09 22:54:03,206 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/table 2023-02-09 22:54:03,206 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 22:54:03,207 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:03,209 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740 2023-02-09 22:54:03,212 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740 2023-02-09 22:54:03,216 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 22:54:03,218 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 22:54:03,219 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65307003, jitterRate=-0.026849821209907532}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 22:54:03,219 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675983243195Writing region info on filesystem at 1675983243195Initializing all the Stores at 1675983243197 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983243197Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983243198 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983243198Cleaning up temporary data from old regions at 1675983243214 (+16 ms)Cleaning up detritus from prior splits at 1675983243214Running coprocessor post-open hooks at 1675983243219 (+5 ms)Region opened successfully at 1675983243219 2023-02-09 22:54:03,229 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675983243101 2023-02-09 22:54:03,246 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 22:54:03,247 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 22:54:03,247 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,44627,1675983241473, state=OPEN 2023-02-09 22:54:03,252 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 22:54:03,252 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 22:54:03,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 22:54:03,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,44627,1675983241473 in 326 msec 2023-02-09 22:54:03,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 22:54:03,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 530 msec 2023-02-09 22:54:03,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 771 msec 2023-02-09 22:54:03,266 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675983241548, completionTime=-1 2023-02-09 22:54:03,267 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-09 22:54:03,267 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:03,333 DEBUG [hconnection-0xaab1b33-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:03,335 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:44700, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:03,353 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:03,353 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983303353 2023-02-09 22:54:03,353 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983363353 2023-02-09 22:54:03,353 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 86 msec 2023-02-09 22:54:03,379 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44215,1675983240249-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:03,380 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44215,1675983240249-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:03,380 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44215,1675983240249-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:03,382 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:44215, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:03,382 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 22:54:03,392 DEBUG [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 22:54:03,402 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 22:54:03,403 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 22:54:03,413 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:03,416 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:03,419 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:03,440 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,443 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13 empty. 2023-02-09 22:54:03,443 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 22:54:03,481 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:03,483 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => f0fb98f20238a9a9ac1a9a9d1be29f13, NAME => 'hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:54:03,517 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:03,517 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing f0fb98f20238a9a9ac1a9a9d1be29f13, disabling compactions & flushes 2023-02-09 22:54:03,517 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:54:03,517 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:54:03,517 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. after waiting 0 ms 2023-02-09 22:54:03,517 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:54:03,518 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:54:03,518 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for f0fb98f20238a9a9ac1a9a9d1be29f13: Waiting for close lock at 1675983243517Disabling compacts and flushes for region at 1675983243517Disabling writes for close at 1675983243517Writing region close event to WAL at 1675983243518 (+1 ms)Closed at 1675983243518 2023-02-09 22:54:03,523 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:03,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675983243526"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983243526"}]},"ts":"1675983243526"} 2023-02-09 22:54:03,566 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:03,568 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:03,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983243569"}]},"ts":"1675983243569"} 2023-02-09 22:54:03,578 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 22:54:03,583 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:03,584 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:03,584 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:03,584 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:03,584 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:03,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f0fb98f20238a9a9ac1a9a9d1be29f13, ASSIGN}] 2023-02-09 22:54:03,590 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f0fb98f20238a9a9ac1a9a9d1be29f13, ASSIGN 2023-02-09 22:54:03,591 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f0fb98f20238a9a9ac1a9a9d1be29f13, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41885,1675983241428; forceNewPlan=false, retain=false 2023-02-09 22:54:03,744 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:03,745 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:03,745 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675983243745"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983243745"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983243745"}]},"ts":"1675983243745"} 2023-02-09 22:54:03,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f0fb98f20238a9a9ac1a9a9d1be29f13, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:03,904 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:03,904 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 22:54:03,911 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:49612, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 22:54:03,917 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:54:03,917 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f0fb98f20238a9a9ac1a9a9d1be29f13, NAME => 'hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:03,918 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:03,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:03,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,919 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,921 INFO [StoreOpener-f0fb98f20238a9a9ac1a9a9d1be29f13-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,923 DEBUG [StoreOpener-f0fb98f20238a9a9ac1a9a9d1be29f13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/info 2023-02-09 22:54:03,923 DEBUG [StoreOpener-f0fb98f20238a9a9ac1a9a9d1be29f13-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/info 2023-02-09 22:54:03,924 INFO [StoreOpener-f0fb98f20238a9a9ac1a9a9d1be29f13-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0fb98f20238a9a9ac1a9a9d1be29f13 columnFamilyName info 2023-02-09 22:54:03,925 INFO [StoreOpener-f0fb98f20238a9a9ac1a9a9d1be29f13-1] regionserver.HStore(310): Store=f0fb98f20238a9a9ac1a9a9d1be29f13/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:03,927 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,928 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,933 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:54:03,938 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:03,939 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened f0fb98f20238a9a9ac1a9a9d1be29f13; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69572592, jitterRate=0.0367124080657959}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:03,939 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for f0fb98f20238a9a9ac1a9a9d1be29f13: Running coprocessor pre-open hook at 1675983243919Writing region info on filesystem at 1675983243919Initializing all the Stores at 1675983243921 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983243921Cleaning up temporary data from old regions at 1675983243929 (+8 ms)Cleaning up detritus from prior splits at 1675983243930 (+1 ms)Running coprocessor post-open hooks at 1675983243939 (+9 ms)Region opened successfully at 1675983243939 2023-02-09 22:54:03,941 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13., pid=6, masterSystemTime=1675983243904 2023-02-09 22:54:03,945 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:54:03,946 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:54:03,947 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:03,948 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675983243947"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983243947"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983243947"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983243947"}]},"ts":"1675983243947"} 2023-02-09 22:54:03,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 22:54:03,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f0fb98f20238a9a9ac1a9a9d1be29f13, server=jenkins-hbase9.apache.org,41885,1675983241428 in 203 msec 2023-02-09 22:54:03,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 22:54:03,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f0fb98f20238a9a9ac1a9a9d1be29f13, ASSIGN in 370 msec 2023-02-09 22:54:03,962 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:03,962 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983243962"}]},"ts":"1675983243962"} 2023-02-09 22:54:03,965 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 22:54:03,968 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:03,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 563 msec 2023-02-09 22:54:04,016 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 22:54:04,019 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 22:54:04,019 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:04,041 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:04,045 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:49620, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:04,062 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:04,082 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 22:54:04,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 38 msec 2023-02-09 22:54:04,097 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:04,108 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 22:54:04,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-09 22:54:04,123 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 22:54:04,126 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 22:54:04,126 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.567sec 2023-02-09 22:54:04,129 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 22:54:04,161 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 22:54:04,162 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 22:54:04,165 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44215,1675983240249-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 22:54:04,167 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,44215,1675983240249-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 22:54:04,177 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 22:54:04,232 DEBUG [Listener at localhost/44743] zookeeper.ReadOnlyZKClient(139): Connect 0x0c8d5aec to 127.0.0.1:49544 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:04,237 DEBUG [Listener at localhost/44743] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ff608fa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:04,252 DEBUG [hconnection-0x4d838b63-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:04,262 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:44710, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:04,273 INFO [Listener at localhost/44743] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:54:04,304 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=362, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=168, AvailableMemoryMB=11247 2023-02-09 22:54:04,308 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:04,317 DEBUG [Listener at localhost/44743] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:54:04,321 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:59126, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:54:04,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:54:04,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:54:04,337 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:54:04,343 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,346 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:04,348 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:04,350 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,351 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3 empty. 2023-02-09 22:54:04,354 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,354 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 22:54:04,381 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:04,383 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => a6ea97c1cacf1b7de3443bbe631266e3, NAME => 'testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:54:04,413 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:04,413 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing a6ea97c1cacf1b7de3443bbe631266e3, disabling compactions & flushes 2023-02-09 22:54:04,413 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,413 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,413 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. after waiting 0 ms 2023-02-09 22:54:04,413 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,413 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,413 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for a6ea97c1cacf1b7de3443bbe631266e3: Waiting for close lock at 1675983244413Disabling compacts and flushes for region at 1675983244413Disabling writes for close at 1675983244413Writing region close event to WAL at 1675983244413Closed at 1675983244413 2023-02-09 22:54:04,418 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:04,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675983244421"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983244421"}]},"ts":"1675983244421"} 2023-02-09 22:54:04,425 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:04,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:04,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983244427"}]},"ts":"1675983244427"} 2023-02-09 22:54:04,430 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-09 22:54:04,435 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:04,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:04,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:04,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:04,437 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:04,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a6ea97c1cacf1b7de3443bbe631266e3, ASSIGN}] 2023-02-09 22:54:04,441 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a6ea97c1cacf1b7de3443bbe631266e3, ASSIGN 2023-02-09 22:54:04,444 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a6ea97c1cacf1b7de3443bbe631266e3, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:54:04,595 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:04,596 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a6ea97c1cacf1b7de3443bbe631266e3, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:04,596 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675983244596"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983244596"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983244596"}]},"ts":"1675983244596"} 2023-02-09 22:54:04,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure a6ea97c1cacf1b7de3443bbe631266e3, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:04,754 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:04,754 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 22:54:04,756 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:47972, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 22:54:04,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6ea97c1cacf1b7de3443bbe631266e3, NAME => 'testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:04,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:04,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:04,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,764 INFO [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,766 DEBUG [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1 2023-02-09 22:54:04,766 DEBUG [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1 2023-02-09 22:54:04,767 INFO [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6ea97c1cacf1b7de3443bbe631266e3 columnFamilyName cf1 2023-02-09 22:54:04,768 INFO [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] regionserver.HStore(310): Store=a6ea97c1cacf1b7de3443bbe631266e3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:04,768 INFO [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,770 DEBUG [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2 2023-02-09 22:54:04,770 DEBUG [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2 2023-02-09 22:54:04,770 INFO [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6ea97c1cacf1b7de3443bbe631266e3 columnFamilyName cf2 2023-02-09 22:54:04,771 INFO [StoreOpener-a6ea97c1cacf1b7de3443bbe631266e3-1] regionserver.HStore(310): Store=a6ea97c1cacf1b7de3443bbe631266e3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:04,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:04,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:04,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:04,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened a6ea97c1cacf1b7de3443bbe631266e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72445979, jitterRate=0.07952921092510223}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:04,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for a6ea97c1cacf1b7de3443bbe631266e3: Running coprocessor pre-open hook at 1675983244763Writing region info on filesystem at 1675983244763Initializing all the Stores at 1675983244764 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983244764Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983244764Cleaning up temporary data from old regions at 1675983244775 (+11 ms)Cleaning up detritus from prior splits at 1675983244775Running coprocessor post-open hooks at 1675983244783 (+8 ms)Region opened successfully at 1675983244783 2023-02-09 22:54:04,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3., pid=11, masterSystemTime=1675983244754 2023-02-09 22:54:04,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a6ea97c1cacf1b7de3443bbe631266e3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:04,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675983244790"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983244790"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983244790"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983244790"}]},"ts":"1675983244790"} 2023-02-09 22:54:04,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-09 22:54:04,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure a6ea97c1cacf1b7de3443bbe631266e3, server=jenkins-hbase9.apache.org,40413,1675983241505 in 193 msec 2023-02-09 22:54:04,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-09 22:54:04,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a6ea97c1cacf1b7de3443bbe631266e3, ASSIGN in 359 msec 2023-02-09 22:54:04,801 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:04,801 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983244801"}]},"ts":"1675983244801"} 2023-02-09 22:54:04,803 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-09 22:54:04,806 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:04,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 465 msec 2023-02-09 22:54:04,848 DEBUG [Listener at localhost/44743] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:04,852 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:47988, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:04,872 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:04,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,880 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:04,880 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:04,880 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 22:54:04,880 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 22:54:04,881 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,881 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,882 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:04,882 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:04,882 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:04,883 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:04,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:04,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:04,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:04,883 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:04,883 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:04,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:04,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:04,885 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,885 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,885 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,886 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,888 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:04,888 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:04,888 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:04,888 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:04,889 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:04,889 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:04,888 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:04,889 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:04,889 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:04,890 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:04,890 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:04,890 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:04,890 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:04,890 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:04,891 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:04,891 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:04,891 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. started... 2023-02-09 22:54:04,892 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6ea97c1cacf1b7de3443bbe631266e3 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:54:04,892 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,892 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,893 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,893 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:04,893 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:04,893 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:04,969 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/aec6450402d345298840d1f0251e014f 2023-02-09 22:54:05,036 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/33722cfc272a43268c4041ef8a2514a4 2023-02-09 22:54:05,046 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/aec6450402d345298840d1f0251e014f as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f 2023-02-09 22:54:05,056 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:05,059 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/33722cfc272a43268c4041ef8a2514a4 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4 2023-02-09 22:54:05,068 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:05,071 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 179ms, sequenceid=6, compaction requested=false 2023-02-09 22:54:05,072 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,074 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6ea97c1cacf1b7de3443bbe631266e3: 2023-02-09 22:54:05,074 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:05,074 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:05,075 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:05,075 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,075 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,075 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,079 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,079 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,079 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 22:54:05,084 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:05,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60ed4a7[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:05,084 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,088 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,089 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:05,089 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:05,089 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:05,093 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,093 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,093 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,096 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,096 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,096 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,096 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,096 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,096 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,097 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:05,101 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 22:54:05,101 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:05,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:05,101 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:05,102 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,102 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:05,104 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,104 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,104 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,104 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,105 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,104 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,105 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,105 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,106 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,107 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,110 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,110 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,111 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,122 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,122 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,122 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,123 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,122 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,123 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,123 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,123 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,123 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,124 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,124 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,125 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:05,125 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:05,226 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:05,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:05,255 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:05,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:05,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,259 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,259 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,260 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 22:54:05,260 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 22:54:05,260 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,260 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,262 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,262 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,262 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,262 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,262 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,262 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,264 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,264 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,264 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,264 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,265 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,264 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,266 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,266 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,266 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,265 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,266 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,266 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,267 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:05,267 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:05,267 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:05,267 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,267 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,267 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. started... 2023-02-09 22:54:05,267 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,267 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6ea97c1cacf1b7de3443bbe631266e3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:05,270 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,271 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,272 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,272 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,272 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,272 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,292 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/2855bedc415e41c2bde61f44e3427d63 2023-02-09 22:54:05,321 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/dac9c59306e04813940a852856bc3ff3 2023-02-09 22:54:05,331 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/2855bedc415e41c2bde61f44e3427d63 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63 2023-02-09 22:54:05,342 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:05,344 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/dac9c59306e04813940a852856bc3ff3 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3 2023-02-09 22:54:05,355 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:05,357 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 90ms, sequenceid=19, compaction requested=false 2023-02-09 22:54:05,357 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6ea97c1cacf1b7de3443bbe631266e3: 2023-02-09 22:54:05,357 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:05,357 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:05,361 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:05,361 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,361 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,361 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,365 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,366 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,366 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,367 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 22:54:05,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@dd96173[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:05,377 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:05,377 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,378 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,378 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,378 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:05,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:05,379 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,380 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:05,379 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,380 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:05,388 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,388 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,388 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,388 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,390 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,390 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,390 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,391 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,391 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,391 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:05,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:05,397 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 22:54:05,397 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:05,397 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:05,397 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,397 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:05,400 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,400 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,401 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,403 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,405 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,408 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,408 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,409 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,424 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,423 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,424 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,424 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,425 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,425 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,425 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,425 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,425 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:05,425 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,425 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:05,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,426 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,426 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,426 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,426 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,526 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:05,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:05,555 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:05,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:05,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,558 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,558 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,559 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 22:54:05,559 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 22:54:05,559 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,560 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,561 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,561 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,561 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,561 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,562 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,562 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,563 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,563 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,563 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,563 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,563 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,564 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,564 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,564 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,563 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,565 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,565 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,564 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,566 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:05,566 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:05,566 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,565 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,566 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:05,566 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,566 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. started... 2023-02-09 22:54:05,566 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6ea97c1cacf1b7de3443bbe631266e3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:05,568 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,569 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,569 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,569 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,570 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,570 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,585 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/c72b200f5b094bdca09cd814583f3956 2023-02-09 22:54:05,612 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/2e727412cbff483aa9180860ca154336 2023-02-09 22:54:05,621 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/c72b200f5b094bdca09cd814583f3956 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956 2023-02-09 22:54:05,628 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:05,630 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/2e727412cbff483aa9180860ca154336 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336 2023-02-09 22:54:05,638 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:05,640 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 73ms, sequenceid=32, compaction requested=true 2023-02-09 22:54:05,640 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6ea97c1cacf1b7de3443bbe631266e3: 2023-02-09 22:54:05,640 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:05,640 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:05,641 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:05,641 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,641 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,641 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,645 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,645 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,645 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 22:54:05,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53730042[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:05,648 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:05,648 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,649 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,649 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,649 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,650 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,650 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:05,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:05,650 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:05,653 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,653 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,653 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,653 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,654 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,654 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,654 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,654 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,654 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,654 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:05,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:05,659 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 22:54:05,659 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:05,659 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:05,659 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,659 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:05,662 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,662 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,662 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,662 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,662 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,662 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,663 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,662 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,663 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,664 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,666 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,667 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,667 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,671 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,671 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,671 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,671 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,672 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,671 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,671 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,673 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,672 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,673 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:05,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,674 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:05,674 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:05,774 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:05,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:05,799 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:05,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:05,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,801 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,801 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 22:54:05,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 22:54:05,803 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,803 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,804 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,805 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,805 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:05,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:05,807 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,807 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,807 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:05,807 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,807 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,808 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,807 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:05,809 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,809 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. started... 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:05,809 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,810 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6ea97c1cacf1b7de3443bbe631266e3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:05,811 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,815 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,815 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,815 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,815 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,815 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,828 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/9c23b612c90e4968b38df77011c55d38 2023-02-09 22:54:05,863 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/ca63364cfaae488585483a3e1ce56772 2023-02-09 22:54:05,873 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/9c23b612c90e4968b38df77011c55d38 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38 2023-02-09 22:54:05,882 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:05,884 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/ca63364cfaae488585483a3e1ce56772 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772 2023-02-09 22:54:05,894 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:05,896 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 86ms, sequenceid=45, compaction requested=true 2023-02-09 22:54:05,896 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6ea97c1cacf1b7de3443bbe631266e3: 2023-02-09 22:54:05,896 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:05,896 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:05,897 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:05,897 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,897 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:05,897 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:05,903 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,903 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,903 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,903 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:05,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 22:54:05,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f65940f[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:05,906 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:05,906 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,907 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,907 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,908 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,907 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,908 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:05,907 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,908 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:05,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,908 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,908 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:05,908 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:05,908 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:05,909 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,909 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,909 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,911 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,911 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,911 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,912 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:05,912 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,912 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,913 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:05,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:05,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:05,917 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 22:54:05,917 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:05,917 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:05,917 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,917 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:05,918 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:05,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,919 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,919 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,920 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:05,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,920 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:05,922 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,923 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,923 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,926 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,926 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,926 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,926 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,927 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:05,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,927 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:05,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:05,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:05,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,928 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:05,928 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:05,928 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:05,928 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:05,929 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,929 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:05,929 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:05,929 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,929 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:05,929 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:05,929 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:05,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:05,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:05,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,029 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:06,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:06,054 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:06,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:06,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,056 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:06,056 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,057 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 22:54:06,057 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 22:54:06,058 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,058 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,060 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,060 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,060 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,060 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,060 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,060 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:06,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:06,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:06,062 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:06,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:06,065 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,064 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,065 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:06,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,065 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:06,065 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,065 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:06,065 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:06,068 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,068 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. started... 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:06,068 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:06,068 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6ea97c1cacf1b7de3443bbe631266e3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:06,071 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,071 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,071 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,071 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:06,071 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,071 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:06,091 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/3069bea8a0424229a5a201719a728c85 2023-02-09 22:54:06,121 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/4ca71860c1644ca8afd240ac1130adfe 2023-02-09 22:54:06,130 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/3069bea8a0424229a5a201719a728c85 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/3069bea8a0424229a5a201719a728c85 2023-02-09 22:54:06,138 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/3069bea8a0424229a5a201719a728c85, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:06,139 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/4ca71860c1644ca8afd240ac1130adfe as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/4ca71860c1644ca8afd240ac1130adfe 2023-02-09 22:54:06,147 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/4ca71860c1644ca8afd240ac1130adfe, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:06,148 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 80ms, sequenceid=58, compaction requested=true 2023-02-09 22:54:06,148 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6ea97c1cacf1b7de3443bbe631266e3: 2023-02-09 22:54:06,148 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,148 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:06,149 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:06,149 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,149 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:06,149 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:06,151 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,152 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,152 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:06,153 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:06,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:06,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:06,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:06,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:06,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 22:54:06,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f3ec8a8[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:06,155 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:06,155 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:06,157 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:06,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,157 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:06,157 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:06,157 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:06,158 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:06,157 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:06,158 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:06,158 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:06,158 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:06,160 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:06,160 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,160 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:06,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,161 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:06,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:06,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:06,161 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:06,161 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,162 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,162 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:06,162 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:06,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:06,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:06,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:06,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:06,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:06,168 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 22:54:06,168 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:06,168 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:06,168 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,168 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:06,169 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,169 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:06,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,170 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,171 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,172 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:06,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,173 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,174 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:06,174 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,181 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,181 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,181 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,181 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,181 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:06,181 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:06,181 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,180 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,181 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,181 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,182 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,182 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:06,182 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,182 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:06,182 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,282 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:06,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:06,303 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:06,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:06,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,305 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:06,305 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,306 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 22:54:06,306 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 22:54:06,306 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,306 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,308 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,308 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,308 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,310 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:06,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,310 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:06,310 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:06,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,310 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,310 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:06,310 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:06,311 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:06,312 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 22:54:06,312 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:06,312 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:06,312 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:06,312 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,312 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. started... 2023-02-09 22:54:06,313 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6ea97c1cacf1b7de3443bbe631266e3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:06,314 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,320 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,321 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,321 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:06,321 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,321 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:06,331 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/77345dcc42964de9859927519e067c42 2023-02-09 22:54:06,356 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/89f6175cd5574a10b585389878ea06a3 2023-02-09 22:54:06,364 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/77345dcc42964de9859927519e067c42 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/77345dcc42964de9859927519e067c42 2023-02-09 22:54:06,371 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/77345dcc42964de9859927519e067c42, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:06,372 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/89f6175cd5574a10b585389878ea06a3 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/89f6175cd5574a10b585389878ea06a3 2023-02-09 22:54:06,379 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/89f6175cd5574a10b585389878ea06a3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:06,380 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 67ms, sequenceid=71, compaction requested=true 2023-02-09 22:54:06,381 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6ea97c1cacf1b7de3443bbe631266e3: 2023-02-09 22:54:06,381 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,381 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:06,382 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:06,382 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,382 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 22:54:06,382 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 22:54:06,384 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,384 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:06,384 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,384 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:06,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:06,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 22:54:06,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69aae6c5[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:06,386 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:06,386 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,387 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:06,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:06,388 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:06,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:06,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 22:54:06,388 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:06,390 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:06,390 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,390 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:06,392 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,392 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:06,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:06,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:06,392 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 22:54:06,392 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,392 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,392 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:06,393 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 22:54:06,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:06,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:06,397 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 22:54:06,397 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:06,397 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:06,397 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,397 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:06,399 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,399 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,399 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,400 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,401 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,401 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,403 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:06,403 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,404 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,407 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 22:54:06,408 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:06,407 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:06,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,409 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:06,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:06,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,407 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:06,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,409 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,409 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:06,409 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:06,409 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:06,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:06,409 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,410 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 22:54:06,509 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:06,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:06,531 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,537 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:06,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:06,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:06,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:06,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a6ea97c1cacf1b7de3443bbe631266e3, UNASSIGN}] 2023-02-09 22:54:06,546 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a6ea97c1cacf1b7de3443bbe631266e3, UNASSIGN 2023-02-09 22:54:06,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a6ea97c1cacf1b7de3443bbe631266e3, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:06,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675983246547"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983246547"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983246547"}]},"ts":"1675983246547"} 2023-02-09 22:54:06,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure a6ea97c1cacf1b7de3443bbe631266e3, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:06,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing a6ea97c1cacf1b7de3443bbe631266e3, disabling compactions & flushes 2023-02-09 22:54:06,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. after waiting 0 ms 2023-02-09 22:54:06,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing a6ea97c1cacf1b7de3443bbe631266e3 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 22:54:06,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/5626e4d7de55447aae3300dbf1af0176 2023-02-09 22:54:06,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/d8ad2e6af5964dfcbb4348981b50907d 2023-02-09 22:54:06,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf1/5626e4d7de55447aae3300dbf1af0176 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/5626e4d7de55447aae3300dbf1af0176 2023-02-09 22:54:06,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/5626e4d7de55447aae3300dbf1af0176, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:54:06,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.tmp/cf2/d8ad2e6af5964dfcbb4348981b50907d as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/d8ad2e6af5964dfcbb4348981b50907d 2023-02-09 22:54:06,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/d8ad2e6af5964dfcbb4348981b50907d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:54:06,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 61ms, sequenceid=82, compaction requested=true 2023-02-09 22:54:06,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 22:54:06,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:06,793 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. 2023-02-09 22:54:06,794 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for a6ea97c1cacf1b7de3443bbe631266e3: Waiting for close lock at 1675983246710Running coprocessor pre-close hooks at 1675983246710Disabling compacts and flushes for region at 1675983246710Disabling writes for close at 1675983246710Obtaining lock to block concurrent updates at 1675983246710Preparing flush snapshotting stores in a6ea97c1cacf1b7de3443bbe631266e3 at 1675983246710Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675983246710Flushing stores of testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. at 1675983246711 (+1 ms)Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf1: creating writer at 1675983246711Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf1: appending metadata at 1675983246714 (+3 ms)Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf1: closing flushed file at 1675983246714Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf2: creating writer at 1675983246733 (+19 ms)Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf2: appending metadata at 1675983246736 (+3 ms)Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf2: closing flushed file at 1675983246736Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf1: reopening flushed file at 1675983246756 (+20 ms)Flushing a6ea97c1cacf1b7de3443bbe631266e3/cf2: reopening flushed file at 1675983246765 (+9 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a6ea97c1cacf1b7de3443bbe631266e3 in 61ms, sequenceid=82, compaction requested=true at 1675983246772 (+7 ms)Writing region close event to WAL at 1675983246788 (+16 ms)Running coprocessor post-close hooks at 1675983246793 (+5 ms)Closed at 1675983246793 2023-02-09 22:54:06,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,799 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a6ea97c1cacf1b7de3443bbe631266e3, regionState=CLOSED 2023-02-09 22:54:06,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675983246799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983246799"}]},"ts":"1675983246799"} 2023-02-09 22:54:06,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-09 22:54:06,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure a6ea97c1cacf1b7de3443bbe631266e3, server=jenkins-hbase9.apache.org,40413,1675983241505 in 254 msec 2023-02-09 22:54:06,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-09 22:54:06,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a6ea97c1cacf1b7de3443bbe631266e3, UNASSIGN in 261 msec 2023-02-09 22:54:06,829 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=a6ea97c1cacf1b7de3443bbe631266e3, threads=14 2023-02-09 22:54:06,830 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,832 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,832 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/4ca71860c1644ca8afd240ac1130adfe for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,832 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/89f6175cd5574a10b585389878ea06a3 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,832 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,832 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/d8ad2e6af5964dfcbb4348981b50907d for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,833 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,833 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,833 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/3069bea8a0424229a5a201719a728c85 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,833 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/5626e4d7de55447aae3300dbf1af0176 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,834 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/77345dcc42964de9859927519e067c42 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,834 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,835 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,835 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,861 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/5626e4d7de55447aae3300dbf1af0176, top=true 2023-02-09 22:54:06,868 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/3069bea8a0424229a5a201719a728c85, top=true 2023-02-09 22:54:06,869 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/77345dcc42964de9859927519e067c42, top=true 2023-02-09 22:54:06,877 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/4ca71860c1644ca8afd240ac1130adfe, top=true 2023-02-09 22:54:06,877 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/89f6175cd5574a10b585389878ea06a3, top=true 2023-02-09 22:54:06,903 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/d8ad2e6af5964dfcbb4348981b50907d, top=true 2023-02-09 22:54:06,911 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.splits/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-89f6175cd5574a10b585389878ea06a3 for child: f43c7c3e652b8b8ace7943731e4b0afe, parent: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,916 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/89f6175cd5574a10b585389878ea06a3 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,918 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.splits/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-4ca71860c1644ca8afd240ac1130adfe for child: f43c7c3e652b8b8ace7943731e4b0afe, parent: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,919 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/4ca71860c1644ca8afd240ac1130adfe for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,919 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.splits/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-3069bea8a0424229a5a201719a728c85 for child: f43c7c3e652b8b8ace7943731e4b0afe, parent: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,920 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/3069bea8a0424229a5a201719a728c85 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,923 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.splits/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-77345dcc42964de9859927519e067c42 for child: f43c7c3e652b8b8ace7943731e4b0afe, parent: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,923 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/77345dcc42964de9859927519e067c42 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,923 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.splits/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-5626e4d7de55447aae3300dbf1af0176 for child: f43c7c3e652b8b8ace7943731e4b0afe, parent: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,923 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/5626e4d7de55447aae3300dbf1af0176 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,926 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/.splits/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-d8ad2e6af5964dfcbb4348981b50907d for child: f43c7c3e652b8b8ace7943731e4b0afe, parent: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:06,926 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/d8ad2e6af5964dfcbb4348981b50907d for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,027 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,028 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,030 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,037 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,041 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,044 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,044 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,349 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63 for region: a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:07,349 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region a6ea97c1cacf1b7de3443bbe631266e3 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:54:07,401 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:54:07,403 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:54:07,405 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675983247405"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675983247405"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675983247405"}]},"ts":"1675983247405"} 2023-02-09 22:54:07,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983247405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983247405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983247405"}]},"ts":"1675983247405"} 2023-02-09 22:54:07,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983247405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983247405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983247405"}]},"ts":"1675983247405"} 2023-02-09 22:54:07,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c4aae79c5625c8879e05e5374437a060, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f43c7c3e652b8b8ace7943731e4b0afe, ASSIGN}] 2023-02-09 22:54:07,454 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f43c7c3e652b8b8ace7943731e4b0afe, ASSIGN 2023-02-09 22:54:07,455 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f43c7c3e652b8b8ace7943731e4b0afe, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:54:07,456 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c4aae79c5625c8879e05e5374437a060, ASSIGN 2023-02-09 22:54:07,457 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c4aae79c5625c8879e05e5374437a060, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:54:07,605 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 22:54:07,607 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c4aae79c5625c8879e05e5374437a060, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:07,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983247607"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983247607"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983247607"}]},"ts":"1675983247607"} 2023-02-09 22:54:07,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure c4aae79c5625c8879e05e5374437a060, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:07,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f43c7c3e652b8b8ace7943731e4b0afe, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:07,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983247614"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983247614"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983247614"}]},"ts":"1675983247614"} 2023-02-09 22:54:07,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure f43c7c3e652b8b8ace7943731e4b0afe, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:07,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:07,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f43c7c3e652b8b8ace7943731e4b0afe, NAME => 'testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.', STARTKEY => '26', ENDKEY => ''} 2023-02-09 22:54:07,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:07,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:07,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,769 INFO [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,770 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1 2023-02-09 22:54:07,770 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1 2023-02-09 22:54:07,771 INFO [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f43c7c3e652b8b8ace7943731e4b0afe columnFamilyName cf1 2023-02-09 22:54:07,802 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63-top 2023-02-09 22:54:07,808 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38-top 2023-02-09 22:54:07,816 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f-top 2023-02-09 22:54:07,823 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956-top 2023-02-09 22:54:07,829 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-3069bea8a0424229a5a201719a728c85 2023-02-09 22:54:07,834 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-5626e4d7de55447aae3300dbf1af0176 2023-02-09 22:54:07,839 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-77345dcc42964de9859927519e067c42 2023-02-09 22:54:07,839 INFO [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(310): Store=f43c7c3e652b8b8ace7943731e4b0afe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:07,840 INFO [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,841 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2 2023-02-09 22:54:07,841 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2 2023-02-09 22:54:07,841 INFO [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f43c7c3e652b8b8ace7943731e4b0afe columnFamilyName cf2 2023-02-09 22:54:07,856 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336-top 2023-02-09 22:54:07,861 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4-top 2023-02-09 22:54:07,867 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772-top 2023-02-09 22:54:07,873 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3-top 2023-02-09 22:54:07,878 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-4ca71860c1644ca8afd240ac1130adfe 2023-02-09 22:54:07,882 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-89f6175cd5574a10b585389878ea06a3 2023-02-09 22:54:07,887 DEBUG [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-d8ad2e6af5964dfcbb4348981b50907d 2023-02-09 22:54:07,887 INFO [StoreOpener-f43c7c3e652b8b8ace7943731e4b0afe-1] regionserver.HStore(310): Store=f43c7c3e652b8b8ace7943731e4b0afe/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:07,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:07,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:07,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened f43c7c3e652b8b8ace7943731e4b0afe; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59095341, jitterRate=-0.11941079795360565}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:07,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for f43c7c3e652b8b8ace7943731e4b0afe: Running coprocessor pre-open hook at 1675983247768Writing region info on filesystem at 1675983247768Initializing all the Stores at 1675983247769 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983247769Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983247769Cleaning up temporary data from old regions at 1675983247891 (+122 ms)Cleaning up detritus from prior splits at 1675983247892 (+1 ms)Running coprocessor post-open hooks at 1675983247895 (+3 ms)Region opened successfully at 1675983247895 2023-02-09 22:54:07,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe., pid=18, masterSystemTime=1675983247762 2023-02-09 22:54:07,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:07,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:07,900 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:07,900 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:07,903 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:07,903 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:07,903 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1912): f43c7c3e652b8b8ace7943731e4b0afe/cf1 is initiating minor compaction (all files) 2023-02-09 22:54:07,903 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1912): f43c7c3e652b8b8ace7943731e4b0afe/cf2 is initiating minor compaction (all files) 2023-02-09 22:54:07,903 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f43c7c3e652b8b8ace7943731e4b0afe/cf1 in testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:07,903 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f43c7c3e652b8b8ace7943731e4b0afe/cf2 in testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:07,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:07,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:07,904 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-3069bea8a0424229a5a201719a728c85, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-77345dcc42964de9859927519e067c42, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-5626e4d7de55447aae3300dbf1af0176] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/.tmp, totalSize=34.9 K 2023-02-09 22:54:07,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:07,904 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f43c7c3e652b8b8ace7943731e4b0afe, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:07,905 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-4ca71860c1644ca8afd240ac1130adfe, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-89f6175cd5574a10b585389878ea06a3, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-d8ad2e6af5964dfcbb4348981b50907d] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/.tmp, totalSize=34.9 K 2023-02-09 22:54:07,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c4aae79c5625c8879e05e5374437a060, NAME => 'testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.', STARTKEY => '', ENDKEY => '26'} 2023-02-09 22:54:07,905 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983247904"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983247904"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983247904"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983247904"}]},"ts":"1675983247904"} 2023-02-09 22:54:07,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:07,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:07,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:07,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:07,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:07,907 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983244853 2023-02-09 22:54:07,907 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983244853 2023-02-09 22:54:07,908 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983245233 2023-02-09 22:54:07,908 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983245233 2023-02-09 22:54:07,909 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983245529 2023-02-09 22:54:07,910 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983245529 2023-02-09 22:54:07,910 INFO [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:07,910 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983245777 2023-02-09 22:54:07,910 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983245777 2023-02-09 22:54:07,911 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-4ca71860c1644ca8afd240ac1130adfe, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983246032 2023-02-09 22:54:07,911 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1 2023-02-09 22:54:07,911 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-3069bea8a0424229a5a201719a728c85, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983246032 2023-02-09 22:54:07,912 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1 2023-02-09 22:54:07,912 INFO [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4aae79c5625c8879e05e5374437a060 columnFamilyName cf1 2023-02-09 22:54:07,912 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-77345dcc42964de9859927519e067c42, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983246284 2023-02-09 22:54:07,913 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-89f6175cd5574a10b585389878ea06a3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983246284 2023-02-09 22:54:07,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-09 22:54:07,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure f43c7c3e652b8b8ace7943731e4b0afe, server=jenkins-hbase9.apache.org,40413,1675983241505 in 293 msec 2023-02-09 22:54:07,918 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-5626e4d7de55447aae3300dbf1af0176, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983246512 2023-02-09 22:54:07,918 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-d8ad2e6af5964dfcbb4348981b50907d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983246512 2023-02-09 22:54:07,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f43c7c3e652b8b8ace7943731e4b0afe, ASSIGN in 465 msec 2023-02-09 22:54:07,936 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63-bottom 2023-02-09 22:54:07,942 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38-bottom 2023-02-09 22:54:07,947 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): f43c7c3e652b8b8ace7943731e4b0afe#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:07,950 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): f43c7c3e652b8b8ace7943731e4b0afe#cf2#compaction#15 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:07,951 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f-bottom 2023-02-09 22:54:07,957 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956-bottom 2023-02-09 22:54:07,959 INFO [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(310): Store=c4aae79c5625c8879e05e5374437a060/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:07,959 INFO [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:07,960 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2 2023-02-09 22:54:07,960 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2 2023-02-09 22:54:07,961 INFO [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4aae79c5625c8879e05e5374437a060 columnFamilyName cf2 2023-02-09 22:54:07,978 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/.tmp/cf2/a17b50d3f4b34309ae500ae7ae1c77b4 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/a17b50d3f4b34309ae500ae7ae1c77b4 2023-02-09 22:54:07,979 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/.tmp/cf1/3c98c98e3f8a4305bef3de7d11c4ee87 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/3c98c98e3f8a4305bef3de7d11c4ee87 2023-02-09 22:54:07,991 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336-bottom 2023-02-09 22:54:07,992 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4-bottom 2023-02-09 22:54:07,997 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772-bottom 2023-02-09 22:54:07,999 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f43c7c3e652b8b8ace7943731e4b0afe/cf2 of f43c7c3e652b8b8ace7943731e4b0afe into a17b50d3f4b34309ae500ae7ae1c77b4(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:08,000 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f43c7c3e652b8b8ace7943731e4b0afe/cf1 of f43c7c3e652b8b8ace7943731e4b0afe into 3c98c98e3f8a4305bef3de7d11c4ee87(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:08,000 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f43c7c3e652b8b8ace7943731e4b0afe: 2023-02-09 22:54:08,000 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f43c7c3e652b8b8ace7943731e4b0afe: 2023-02-09 22:54:08,001 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe., storeName=f43c7c3e652b8b8ace7943731e4b0afe/cf2, priority=9, startTime=1675983247900; duration=0sec 2023-02-09 22:54:08,001 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe., storeName=f43c7c3e652b8b8ace7943731e4b0afe/cf1, priority=9, startTime=1675983247898; duration=0sec 2023-02-09 22:54:08,001 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:08,001 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:08,004 DEBUG [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3-bottom 2023-02-09 22:54:08,004 INFO [StoreOpener-c4aae79c5625c8879e05e5374437a060-1] regionserver.HStore(310): Store=c4aae79c5625c8879e05e5374437a060/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:08,005 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:08,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:08,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:08,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:08,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened c4aae79c5625c8879e05e5374437a060; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74715597, jitterRate=0.11334915459156036}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:08,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for c4aae79c5625c8879e05e5374437a060: Running coprocessor pre-open hook at 1675983247906Writing region info on filesystem at 1675983247906Initializing all the Stores at 1675983247909 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983247909Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983247909Cleaning up temporary data from old regions at 1675983248009 (+100 ms)Cleaning up detritus from prior splits at 1675983248009Running coprocessor post-open hooks at 1675983248012 (+3 ms)Region opened successfully at 1675983248012 2023-02-09 22:54:08,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060., pid=17, masterSystemTime=1675983247762 2023-02-09 22:54:08,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:08,014 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 22:54:08,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:08,014 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 22:54:08,014 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,014 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1912): c4aae79c5625c8879e05e5374437a060/cf1 is initiating minor compaction (all files) 2023-02-09 22:54:08,014 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,014 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c4aae79c5625c8879e05e5374437a060/cf1 in testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,014 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1912): c4aae79c5625c8879e05e5374437a060/cf2 is initiating minor compaction (all files) 2023-02-09 22:54:08,015 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c4aae79c5625c8879e05e5374437a060/cf2 in testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,015 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/.tmp, totalSize=19.9 K 2023-02-09 22:54:08,015 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/.tmp, totalSize=19.9 K 2023-02-09 22:54:08,016 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983244853 2023-02-09 22:54:08,016 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983244853 2023-02-09 22:54:08,016 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983245233 2023-02-09 22:54:08,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,017 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983245233 2023-02-09 22:54:08,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c4aae79c5625c8879e05e5374437a060, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:08,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983248017"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983248017"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983248017"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983248017"}]},"ts":"1675983248017"} 2023-02-09 22:54:08,018 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983245529 2023-02-09 22:54:08,019 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983245529 2023-02-09 22:54:08,019 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983245777 2023-02-09 22:54:08,020 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983245777 2023-02-09 22:54:08,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-09 22:54:08,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure c4aae79c5625c8879e05e5374437a060, server=jenkins-hbase9.apache.org,40413,1675983241505 in 411 msec 2023-02-09 22:54:08,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-09 22:54:08,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c4aae79c5625c8879e05e5374437a060, ASSIGN in 572 msec 2023-02-09 22:54:08,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe in 1.4980 sec 2023-02-09 22:54:08,035 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): c4aae79c5625c8879e05e5374437a060#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:08,035 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c4aae79c5625c8879e05e5374437a060#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:08,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40413] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.2.10:47988 deadline: 1675983308043, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3. is not online on jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:08,056 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/.tmp/cf1/6996d87a43114e13b42a108e9d389992 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/6996d87a43114e13b42a108e9d389992 2023-02-09 22:54:08,057 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/.tmp/cf2/10a2682f585b4eda9cb960b33e57e9b2 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/10a2682f585b4eda9cb960b33e57e9b2 2023-02-09 22:54:08,063 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c4aae79c5625c8879e05e5374437a060/cf1 of c4aae79c5625c8879e05e5374437a060 into 6996d87a43114e13b42a108e9d389992(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:08,063 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c4aae79c5625c8879e05e5374437a060: 2023-02-09 22:54:08,063 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060., storeName=c4aae79c5625c8879e05e5374437a060/cf1, priority=12, startTime=1675983248013; duration=0sec 2023-02-09 22:54:08,063 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:08,064 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c4aae79c5625c8879e05e5374437a060/cf2 of c4aae79c5625c8879e05e5374437a060 into 10a2682f585b4eda9cb960b33e57e9b2(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:08,064 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c4aae79c5625c8879e05e5374437a060: 2023-02-09 22:54:08,064 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060., storeName=c4aae79c5625c8879e05e5374437a060/cf2, priority=12, startTime=1675983248014; duration=0sec 2023-02-09 22:54:08,064 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:08,162 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 22:54:08,179 INFO [Listener at localhost/44743] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 22:54:08,182 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 22:54:08,243 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:08,254 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,258 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983248272"}]},"ts":"1675983248272"} 2023-02-09 22:54:08,274 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-09 22:54:08,276 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-09 22:54:08,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c4aae79c5625c8879e05e5374437a060, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f43c7c3e652b8b8ace7943731e4b0afe, UNASSIGN}] 2023-02-09 22:54:08,279 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f43c7c3e652b8b8ace7943731e4b0afe, UNASSIGN 2023-02-09 22:54:08,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:54:08,280 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=f43c7c3e652b8b8ace7943731e4b0afe, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:08,280 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983248280"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983248280"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983248280"}]},"ts":"1675983248280"} 2023-02-09 22:54:08,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure f43c7c3e652b8b8ace7943731e4b0afe, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:08,287 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c4aae79c5625c8879e05e5374437a060, UNASSIGN 2023-02-09 22:54:08,288 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c4aae79c5625c8879e05e5374437a060, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:08,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983248288"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983248288"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983248288"}]},"ts":"1675983248288"} 2023-02-09 22:54:08,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure c4aae79c5625c8879e05e5374437a060, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:08,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:54:08,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:08,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing f43c7c3e652b8b8ace7943731e4b0afe, disabling compactions & flushes 2023-02-09 22:54:08,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:08,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:08,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. after waiting 0 ms 2023-02-09 22:54:08,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:08,452 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-3069bea8a0424229a5a201719a728c85, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-77345dcc42964de9859927519e067c42, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-5626e4d7de55447aae3300dbf1af0176] to archive 2023-02-09 22:54:08,453 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:08,460 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,462 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,465 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,466 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,468 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-3069bea8a0424229a5a201719a728c85 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-3069bea8a0424229a5a201719a728c85 2023-02-09 22:54:08,470 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-77345dcc42964de9859927519e067c42 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-77345dcc42964de9859927519e067c42 2023-02-09 22:54:08,472 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-5626e4d7de55447aae3300dbf1af0176 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-5626e4d7de55447aae3300dbf1af0176 2023-02-09 22:54:08,492 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-4ca71860c1644ca8afd240ac1130adfe, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-89f6175cd5574a10b585389878ea06a3, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-d8ad2e6af5964dfcbb4348981b50907d] to archive 2023-02-09 22:54:08,493 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:08,495 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,497 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,499 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,501 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,502 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-4ca71860c1644ca8afd240ac1130adfe to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-4ca71860c1644ca8afd240ac1130adfe 2023-02-09 22:54:08,504 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-89f6175cd5574a10b585389878ea06a3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-89f6175cd5574a10b585389878ea06a3 2023-02-09 22:54:08,506 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-d8ad2e6af5964dfcbb4348981b50907d to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/testSplitTableRegionUnevenDaughter=a6ea97c1cacf1b7de3443bbe631266e3-d8ad2e6af5964dfcbb4348981b50907d 2023-02-09 22:54:08,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:54:08,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:08,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe. 2023-02-09 22:54:08,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for f43c7c3e652b8b8ace7943731e4b0afe: Waiting for close lock at 1675983248437Running coprocessor pre-close hooks at 1675983248437Disabling compacts and flushes for region at 1675983248437Disabling writes for close at 1675983248437Writing region close event to WAL at 1675983248507 (+70 ms)Running coprocessor post-close hooks at 1675983248514 (+7 ms)Closed at 1675983248514 2023-02-09 22:54:08,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:08,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:08,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing c4aae79c5625c8879e05e5374437a060, disabling compactions & flushes 2023-02-09 22:54:08,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. after waiting 0 ms 2023-02-09 22:54:08,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,519 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=f43c7c3e652b8b8ace7943731e4b0afe, regionState=CLOSED 2023-02-09 22:54:08,519 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983248519"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983248519"}]},"ts":"1675983248519"} 2023-02-09 22:54:08,520 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38-bottom] to archive 2023-02-09 22:54:08,522 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:08,527 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/aec6450402d345298840d1f0251e014f.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,532 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/2855bedc415e41c2bde61f44e3427d63.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-09 22:54:08,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure f43c7c3e652b8b8ace7943731e4b0afe, server=jenkins-hbase9.apache.org,40413,1675983241505 in 240 msec 2023-02-09 22:54:08,534 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/c72b200f5b094bdca09cd814583f3956.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f43c7c3e652b8b8ace7943731e4b0afe, UNASSIGN in 256 msec 2023-02-09 22:54:08,536 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/9c23b612c90e4968b38df77011c55d38.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,540 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772-bottom] to archive 2023-02-09 22:54:08,541 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:08,545 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/33722cfc272a43268c4041ef8a2514a4.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,547 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/dac9c59306e04813940a852856bc3ff3.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,549 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/2e727412cbff483aa9180860ca154336.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,551 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/ca63364cfaae488585483a3e1ce56772.a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:54:08,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:08,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060. 2023-02-09 22:54:08,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for c4aae79c5625c8879e05e5374437a060: Waiting for close lock at 1675983248518Running coprocessor pre-close hooks at 1675983248518Disabling compacts and flushes for region at 1675983248518Disabling writes for close at 1675983248518Writing region close event to WAL at 1675983248555 (+37 ms)Running coprocessor post-close hooks at 1675983248562 (+7 ms)Closed at 1675983248563 (+1 ms) 2023-02-09 22:54:08,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:08,567 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c4aae79c5625c8879e05e5374437a060, regionState=CLOSED 2023-02-09 22:54:08,567 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983248567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983248567"}]},"ts":"1675983248567"} 2023-02-09 22:54:08,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-09 22:54:08,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure c4aae79c5625c8879e05e5374437a060, server=jenkins-hbase9.apache.org,40413,1675983241505 in 279 msec 2023-02-09 22:54:08,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-09 22:54:08,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c4aae79c5625c8879e05e5374437a060, UNASSIGN in 296 msec 2023-02-09 22:54:08,578 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983248578"}]},"ts":"1675983248578"} 2023-02-09 22:54:08,580 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-09 22:54:08,582 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-09 22:54:08,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 323 msec 2023-02-09 22:54:08,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:54:08,587 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-09 22:54:08,592 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,601 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,603 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 22:54:08,611 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,612 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:08,612 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:08,616 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/recovered.edits] 2023-02-09 22:54:08,616 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/recovered.edits] 2023-02-09 22:54:08,616 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/recovered.edits] 2023-02-09 22:54:08,621 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/2855bedc415e41c2bde61f44e3427d63 2023-02-09 22:54:08,622 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/3c98c98e3f8a4305bef3de7d11c4ee87 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf1/3c98c98e3f8a4305bef3de7d11c4ee87 2023-02-09 22:54:08,623 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/6996d87a43114e13b42a108e9d389992 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf1/6996d87a43114e13b42a108e9d389992 2023-02-09 22:54:08,624 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/3069bea8a0424229a5a201719a728c85 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/3069bea8a0424229a5a201719a728c85 2023-02-09 22:54:08,629 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/5626e4d7de55447aae3300dbf1af0176 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/5626e4d7de55447aae3300dbf1af0176 2023-02-09 22:54:08,629 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/a17b50d3f4b34309ae500ae7ae1c77b4 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/cf2/a17b50d3f4b34309ae500ae7ae1c77b4 2023-02-09 22:54:08,630 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/10a2682f585b4eda9cb960b33e57e9b2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/cf2/10a2682f585b4eda9cb960b33e57e9b2 2023-02-09 22:54:08,631 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/77345dcc42964de9859927519e067c42 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/77345dcc42964de9859927519e067c42 2023-02-09 22:54:08,634 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe/recovered.edits/90.seqid 2023-02-09 22:54:08,634 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/9c23b612c90e4968b38df77011c55d38 2023-02-09 22:54:08,635 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:08,635 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060/recovered.edits/90.seqid 2023-02-09 22:54:08,636 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/aec6450402d345298840d1f0251e014f 2023-02-09 22:54:08,636 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/c4aae79c5625c8879e05e5374437a060 2023-02-09 22:54:08,638 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf1/c72b200f5b094bdca09cd814583f3956 2023-02-09 22:54:08,641 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/2e727412cbff483aa9180860ca154336 2023-02-09 22:54:08,643 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/33722cfc272a43268c4041ef8a2514a4 2023-02-09 22:54:08,645 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/4ca71860c1644ca8afd240ac1130adfe to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/4ca71860c1644ca8afd240ac1130adfe 2023-02-09 22:54:08,646 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/89f6175cd5574a10b585389878ea06a3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/89f6175cd5574a10b585389878ea06a3 2023-02-09 22:54:08,648 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/ca63364cfaae488585483a3e1ce56772 2023-02-09 22:54:08,649 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/d8ad2e6af5964dfcbb4348981b50907d to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/d8ad2e6af5964dfcbb4348981b50907d 2023-02-09 22:54:08,651 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/cf2/dac9c59306e04813940a852856bc3ff3 2023-02-09 22:54:08,655 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/recovered.edits/85.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3/recovered.edits/85.seqid 2023-02-09 22:54:08,655 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionUnevenDaughter/a6ea97c1cacf1b7de3443bbe631266e3 2023-02-09 22:54:08,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 22:54:08,659 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,687 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-09 22:54:08,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 22:54:08,737 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:08,740 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-09 22:54:08,742 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,742 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-09 22:54:08,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983248742"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:08,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983248742"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:08,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983248742"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:08,745 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:54:08,746 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a6ea97c1cacf1b7de3443bbe631266e3, NAME => 'testSplitTableRegionUnevenDaughter,,1675983244341.a6ea97c1cacf1b7de3443bbe631266e3.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c4aae79c5625c8879e05e5374437a060, NAME => 'testSplitTableRegionUnevenDaughter,,1675983246531.c4aae79c5625c8879e05e5374437a060.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => f43c7c3e652b8b8ace7943731e4b0afe, NAME => 'testSplitTableRegionUnevenDaughter,26,1675983246531.f43c7c3e652b8b8ace7943731e4b0afe.', STARTKEY => '26', ENDKEY => ''}] 2023-02-09 22:54:08,746 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-09 22:54:08,746 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983248746"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:08,748 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-09 22:54:08,751 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:08,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 158 msec 2023-02-09 22:54:08,813 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 22:54:08,814 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 22:54:08,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 22:54:08,908 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-09 22:54:08,925 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=395 (was 362) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44626 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44870 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xaab1b33-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44614 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:45466 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:45778 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xaab1b33-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44678 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44906 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:45764 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44644 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:40396 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xaab1b33-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44664 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase9:40413-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:40416 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:45536 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:44922 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:40384 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:40380 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 229), ProcessCount=168 (was 168), AvailableMemoryMB=11190 (was 11247) 2023-02-09 22:54:08,939 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=210, ProcessCount=168, AvailableMemoryMB=11188 2023-02-09 22:54:08,940 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:08,946 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:54:08,946 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:54:08,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:08,950 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:08,951 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:08,953 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:08,953 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b empty. 2023-02-09 22:54:08,954 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:08,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 22:54:08,965 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:08,967 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => c0d992ad86816a23e23291a2fe7d092b, NAME => 'testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:54:08,975 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:08,976 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing c0d992ad86816a23e23291a2fe7d092b, disabling compactions & flushes 2023-02-09 22:54:08,976 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:08,976 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:08,976 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. after waiting 0 ms 2023-02-09 22:54:08,976 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:08,976 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:08,976 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for c0d992ad86816a23e23291a2fe7d092b: Waiting for close lock at 1675983248976Disabling compacts and flushes for region at 1675983248976Disabling writes for close at 1675983248976Writing region close event to WAL at 1675983248976Closed at 1675983248976 2023-02-09 22:54:08,979 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:08,980 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983248979"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983248979"}]},"ts":"1675983248979"} 2023-02-09 22:54:08,982 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:08,983 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:08,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983248983"}]},"ts":"1675983248983"} 2023-02-09 22:54:08,985 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-09 22:54:08,988 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:08,989 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:08,989 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:08,989 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:08,989 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:08,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, ASSIGN}] 2023-02-09 22:54:08,991 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, ASSIGN 2023-02-09 22:54:08,992 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=false 2023-02-09 22:54:09,142 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:09,144 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=c0d992ad86816a23e23291a2fe7d092b, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,144 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983249144"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983249144"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983249144"}]},"ts":"1675983249144"} 2023-02-09 22:54:09,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:09,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c0d992ad86816a23e23291a2fe7d092b, NAME => 'testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:09,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:09,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:09,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,305 INFO [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,307 DEBUG [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1 2023-02-09 22:54:09,307 DEBUG [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1 2023-02-09 22:54:09,307 INFO [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0d992ad86816a23e23291a2fe7d092b columnFamilyName cf1 2023-02-09 22:54:09,308 INFO [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] regionserver.HStore(310): Store=c0d992ad86816a23e23291a2fe7d092b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:09,308 INFO [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,310 DEBUG [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2 2023-02-09 22:54:09,310 DEBUG [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2 2023-02-09 22:54:09,312 INFO [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0d992ad86816a23e23291a2fe7d092b columnFamilyName cf2 2023-02-09 22:54:09,312 INFO [StoreOpener-c0d992ad86816a23e23291a2fe7d092b-1] regionserver.HStore(310): Store=c0d992ad86816a23e23291a2fe7d092b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:09,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:09,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:09,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:09,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened c0d992ad86816a23e23291a2fe7d092b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72615606, jitterRate=0.08205685019493103}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:09,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for c0d992ad86816a23e23291a2fe7d092b: Running coprocessor pre-open hook at 1675983249303Writing region info on filesystem at 1675983249304 (+1 ms)Initializing all the Stores at 1675983249304Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983249304Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983249305 (+1 ms)Cleaning up temporary data from old regions at 1675983249315 (+10 ms)Cleaning up detritus from prior splits at 1675983249315Running coprocessor post-open hooks at 1675983249324 (+9 ms)Region opened successfully at 1675983249324 2023-02-09 22:54:09,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b., pid=27, masterSystemTime=1675983249298 2023-02-09 22:54:09,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,328 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=c0d992ad86816a23e23291a2fe7d092b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,328 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983249328"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983249328"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983249328"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983249328"}]},"ts":"1675983249328"} 2023-02-09 22:54:09,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-09 22:54:09,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 in 184 msec 2023-02-09 22:54:09,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-09 22:54:09,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, ASSIGN in 344 msec 2023-02-09 22:54:09,337 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:09,337 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983249337"}]},"ts":"1675983249337"} 2023-02-09 22:54:09,339 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-09 22:54:09,342 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:09,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 395 msec 2023-02-09 22:54:09,459 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:09,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 22:54:09,462 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 22:54:09,462 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,463 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 22:54:09,463 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 22:54:09,464 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,464 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,466 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,467 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,467 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:09,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,467 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,469 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,469 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,469 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,469 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,469 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,470 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,471 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,471 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,469 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,471 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:09,471 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:09,471 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,471 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,471 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. started... 2023-02-09 22:54:09,471 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,472 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,472 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0d992ad86816a23e23291a2fe7d092b 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:54:09,472 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,472 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,473 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,474 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,474 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,474 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,475 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,475 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,483 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/231ae60a162048d6a3f4b7608da116ad 2023-02-09 22:54:09,498 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/bb901fac0d0048d58bd40d9ae7b819ff 2023-02-09 22:54:09,504 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/231ae60a162048d6a3f4b7608da116ad as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad 2023-02-09 22:54:09,509 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:09,510 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/bb901fac0d0048d58bd40d9ae7b819ff as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff 2023-02-09 22:54:09,515 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:09,516 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 44ms, sequenceid=6, compaction requested=false 2023-02-09 22:54:09,516 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-09 22:54:09,516 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0d992ad86816a23e23291a2fe7d092b: 2023-02-09 22:54:09,516 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,516 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:09,516 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:09,516 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,516 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,516 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,518 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,518 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,518 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 22:54:09,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22908aa2[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:09,521 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 22:54:09,521 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,523 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,523 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,523 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,524 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,524 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:09,524 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:09,526 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,526 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,526 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,527 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,527 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,527 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,527 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,527 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,527 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,528 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 22:54:09,532 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 22:54:09,532 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:09,532 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:09,532 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 22:54:09,532 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:09,534 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,535 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,535 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,536 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,538 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,538 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,539 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,543 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,543 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,543 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 22:54:09,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,544 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,545 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:09,545 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,545 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,545 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,545 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,545 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,545 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,545 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,546 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:09,546 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:09,646 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:09,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:09,665 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:09,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:09,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 22:54:09,667 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 22:54:09,668 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,668 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 22:54:09,668 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 22:54:09,669 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,669 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,670 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,670 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,670 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,670 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,671 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,671 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:09,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:09,672 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:09,673 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,673 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,673 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,673 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,673 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,674 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,674 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,673 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,674 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,674 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,675 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,675 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,675 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:09,675 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:09,675 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,675 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,675 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. started... 2023-02-09 22:54:09,675 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0d992ad86816a23e23291a2fe7d092b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:09,676 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,677 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,677 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,677 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,678 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,678 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,686 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/40236328589d4c58af264fba8cd36b49 2023-02-09 22:54:09,704 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/1e80ec5a4cf04a09ae932450776f1f98 2023-02-09 22:54:09,709 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/40236328589d4c58af264fba8cd36b49 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49 2023-02-09 22:54:09,714 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:09,715 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/1e80ec5a4cf04a09ae932450776f1f98 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98 2023-02-09 22:54:09,719 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:09,720 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 45ms, sequenceid=19, compaction requested=false 2023-02-09 22:54:09,720 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0d992ad86816a23e23291a2fe7d092b: 2023-02-09 22:54:09,720 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,720 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:09,720 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:09,720 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,720 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,720 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,723 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,723 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,724 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,724 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 22:54:09,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51a6a713[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:09,726 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 22:54:09,726 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,727 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,727 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,727 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:09,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,727 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:09,727 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,728 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,728 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:09,728 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,728 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,728 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:09,729 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,729 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,729 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,729 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,734 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,734 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,734 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,735 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,735 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,735 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:54:09,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 22:54:09,739 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 22:54:09,739 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:09,739 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:09,739 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 22:54:09,739 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:09,741 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,741 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,741 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,742 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,742 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,742 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,743 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,745 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,746 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,746 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,750 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,750 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,749 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,750 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 22:54:09,751 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:09,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,752 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:09,752 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:09,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,752 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,752 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,752 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,852 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:09,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:09,871 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:09,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:09,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 22:54:09,873 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 22:54:09,873 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,874 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 22:54:09,874 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 22:54:09,875 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,875 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,876 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,876 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,876 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,876 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,876 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,877 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:09,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:09,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:09,878 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,878 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,879 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,879 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,879 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,879 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:09,879 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,879 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,879 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:09,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:09,880 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. started... 2023-02-09 22:54:09,880 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:09,880 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:09,880 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:09,880 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0d992ad86816a23e23291a2fe7d092b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:09,880 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,880 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,880 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:09,881 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,881 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,882 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,883 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,883 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,883 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,883 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,883 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,892 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/2f52cfa283cf47b99ae5c9885682ea07 2023-02-09 22:54:09,909 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/fd613ae8690648efa8637dbb144b8521 2023-02-09 22:54:09,915 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/2f52cfa283cf47b99ae5c9885682ea07 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07 2023-02-09 22:54:09,921 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:09,921 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/fd613ae8690648efa8637dbb144b8521 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521 2023-02-09 22:54:09,926 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:09,927 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 47ms, sequenceid=32, compaction requested=true 2023-02-09 22:54:09,927 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0d992ad86816a23e23291a2fe7d092b: 2023-02-09 22:54:09,927 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:09,927 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:09,927 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:09,927 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,927 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:09,927 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:09,929 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,930 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,930 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,930 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 22:54:09,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8044e07[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:09,932 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 22:54:09,932 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,933 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,933 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,934 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:09,934 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,933 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,934 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:09,934 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:09,934 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:09,934 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:09,935 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,935 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,935 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,936 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,936 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,936 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:09,936 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,936 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,937 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,937 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:09,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:54:09,941 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 22:54:09,941 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 22:54:09,941 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:09,941 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:09,941 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 22:54:09,941 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:09,943 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,945 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,945 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,946 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:09,948 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,948 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,949 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,952 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,952 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 22:54:09,953 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:09,953 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:09,953 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,953 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:09,953 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:09,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,953 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,952 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,953 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:09,954 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:09,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,954 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,954 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:09,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,954 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,053 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:10,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:10,073 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:10,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:10,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 22:54:10,075 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 22:54:10,075 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,076 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 22:54:10,076 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 22:54:10,076 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,076 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,077 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,077 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,077 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,077 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,078 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,078 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:10,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,079 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,079 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,081 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,081 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,082 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,082 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,082 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,082 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,082 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:10,083 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:10,083 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,083 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,083 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,083 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,082 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:10,083 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,083 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. started... 2023-02-09 22:54:10,083 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,083 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,085 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,085 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,085 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,085 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0d992ad86816a23e23291a2fe7d092b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:10,089 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,090 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,090 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,090 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,091 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,091 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,096 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/ce7e0ba501254a2ba773d9afe11b3388 2023-02-09 22:54:10,110 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/7c934e776f20418c98676a7d5c97f74c 2023-02-09 22:54:10,115 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/ce7e0ba501254a2ba773d9afe11b3388 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388 2023-02-09 22:54:10,120 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:10,120 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/7c934e776f20418c98676a7d5c97f74c as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c 2023-02-09 22:54:10,125 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:10,126 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 43ms, sequenceid=45, compaction requested=true 2023-02-09 22:54:10,126 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0d992ad86816a23e23291a2fe7d092b: 2023-02-09 22:54:10,126 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:10,126 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:10,126 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:10,126 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,126 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,126 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,128 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,128 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,128 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,128 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 22:54:10,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36a61d0c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:10,130 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 22:54:10,131 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,132 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:10,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:10,132 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:10,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,136 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,136 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,136 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,136 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,136 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,136 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,137 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:54:10,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 22:54:10,141 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 22:54:10,142 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:10,142 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:10,142 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 22:54:10,142 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:10,143 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,143 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,143 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,143 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,144 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,145 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,147 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,147 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,147 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,150 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,150 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,150 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,150 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,150 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 22:54:10,150 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:10,150 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,150 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,151 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,150 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,150 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,151 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:10,151 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,151 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,151 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,151 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,151 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,152 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,152 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,152 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,152 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,152 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,252 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:10,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:10,271 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:10,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:10,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 22:54:10,273 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 22:54:10,273 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,273 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 22:54:10,273 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 22:54:10,274 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,274 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,276 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,276 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,276 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,276 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,277 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,277 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:10,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:10,279 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:10,279 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. started... 2023-02-09 22:54:10,280 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0d992ad86816a23e23291a2fe7d092b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:10,281 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,281 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,281 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,282 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,282 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,282 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,290 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/472f8708d9c54b3bb24b0c944d5df1ee 2023-02-09 22:54:10,305 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 2023-02-09 22:54:10,311 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/472f8708d9c54b3bb24b0c944d5df1ee as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee 2023-02-09 22:54:10,317 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:10,318 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 2023-02-09 22:54:10,327 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:10,328 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 48ms, sequenceid=58, compaction requested=true 2023-02-09 22:54:10,329 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0d992ad86816a23e23291a2fe7d092b: 2023-02-09 22:54:10,329 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:10,329 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:10,329 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:10,329 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,329 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,329 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,331 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,331 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,331 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,331 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 22:54:10,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10c544e0[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:10,334 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 22:54:10,334 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,336 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,337 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:10,336 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,336 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,340 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,340 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,340 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,340 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:54:10,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 22:54:10,344 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 22:54:10,344 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:10,344 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:10,344 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 22:54:10,344 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:10,345 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,345 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,345 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,345 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,345 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,345 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,346 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,346 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,347 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,347 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,349 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,349 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,350 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,360 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,360 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 22:54:10,360 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:10,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,360 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,361 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,361 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,361 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:10,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,361 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,361 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,362 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,362 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,362 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,361 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:10,413 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 22:54:10,413 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-09 22:54:10,414 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:10,414 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-09 22:54:10,462 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:10,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:10,484 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:10,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:10,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 22:54:10,486 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 22:54:10,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 22:54:10,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 22:54:10,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,489 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,489 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,489 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:10,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 22:54:10,491 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,491 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,491 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 22:54:10,491 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 22:54:10,491 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,491 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:10,492 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:10,492 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:10,492 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. started... 2023-02-09 22:54:10,492 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c0d992ad86816a23e23291a2fe7d092b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:10,493 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,493 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,494 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,494 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,494 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,494 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,510 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/4f9a534fdf5e4d5e981d6dff142702a6 2023-02-09 22:54:10,530 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/f1235bdef7f0484ebaabf1b9f790d261 2023-02-09 22:54:10,535 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/4f9a534fdf5e4d5e981d6dff142702a6 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6 2023-02-09 22:54:10,541 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:10,542 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/f1235bdef7f0484ebaabf1b9f790d261 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261 2023-02-09 22:54:10,546 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:10,547 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 55ms, sequenceid=71, compaction requested=true 2023-02-09 22:54:10,547 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c0d992ad86816a23e23291a2fe7d092b: 2023-02-09 22:54:10,547 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:10,547 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:10,547 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:10,547 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,547 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 22:54:10,547 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 22:54:10,549 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,549 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,550 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,550 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 22:54:10,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74bff7d4[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:10,552 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 22:54:10,552 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,554 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,555 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,555 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:10,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,554 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:10,555 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 22:54:10,555 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 22:54:10,555 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:10,554 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,555 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:10,556 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,556 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,556 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,556 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,557 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,557 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,557 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 22:54:10,557 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,557 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,557 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 22:54:10,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:54:10,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 22:54:10,560 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 22:54:10,560 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:10,560 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:10,560 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 22:54:10,560 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:10,562 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,562 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,562 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,563 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,564 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,565 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 22:54:10,565 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,566 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,569 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,569 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,569 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,569 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,569 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,569 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,570 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,569 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 22:54:10,570 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:10,569 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,569 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,570 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,570 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:10,570 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,570 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:10,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,571 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 22:54:10,571 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,571 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,571 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:10,571 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:10,571 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 22:54:10,571 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 22:54:10,671 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:10,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:10,933 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 22:54:10,934 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:10,936 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:10,937 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:10,937 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:10,937 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:10,938 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:10,938 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:10,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:10,938 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:11,186 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,187 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:11,187 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:11,188 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:11,189 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-09 22:54:11,190 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:11,190 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:11,190 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:11,190 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:11,191 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:11,195 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 22:54:11,197 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:11,197 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:11,198 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:11,198 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:11,199 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,199 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:11,200 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,200 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,201 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:11,202 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,202 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:11,202 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,202 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,202 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,203 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-09 22:54:11,204 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:11,204 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:11,205 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983281205 2023-02-09 22:54:11,206 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:11,206 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,211 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:11,211 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:11,211 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:11,211 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983311211 2023-02-09 22:54:11,211 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983371211 2023-02-09 22:54:11,211 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:54:11,211 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:11,214 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:11,214 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:11,214 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:11,462 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,462 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:11,462 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:11,462 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:11,463 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:11,463 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:11,463 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:11,463 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:11,463 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:11,464 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:11,466 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 22:54:11,466 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:11,467 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:11,467 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:11,467 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:11,468 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,468 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:11,468 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,469 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,469 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:11,470 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,470 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:11,470 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,470 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,470 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,470 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:54:11,472 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:11,472 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:11,473 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983281473 2023-02-09 22:54:11,474 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,474 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:11,477 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:11,478 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:11,478 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:11,478 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983311478 2023-02-09 22:54:11,478 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983371478 2023-02-09 22:54:11,478 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:11,478 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:11,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN}] 2023-02-09 22:54:11,483 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:11,483 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:11,483 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:11,728 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,728 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:11,729 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:11,729 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:11,729 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:11,729 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:11,729 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:11,729 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:11,729 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:11,730 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:11,731 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 22:54:11,731 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:11,732 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:11,732 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:11,732 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:11,733 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,734 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:11,734 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,734 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,734 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:11,735 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,735 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:11,735 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,735 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,735 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,735 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:11,736 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:11,736 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:11,736 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983281736 2023-02-09 22:54:11,738 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:11,738 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,740 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:11,741 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:11,741 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:11,741 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983311741 2023-02-09 22:54:11,741 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983371741 2023-02-09 22:54:11,741 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:11,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN}] 2023-02-09 22:54:11,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:11,743 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:11,744 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:11,744 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983251744"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983251744"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983251744"}]},"ts":"1675983251744"} 2023-02-09 22:54:11,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:11,746 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:11,746 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:11,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:11,746 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:11,991 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,992 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:11,992 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:11,992 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:11,992 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:11,992 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:11,992 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:11,992 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:11,992 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:11,993 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:11,994 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:11,994 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 22:54:11,995 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:11,995 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:11,995 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:11,996 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:11,996 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,996 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:11,997 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,997 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:11,997 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:11,998 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,998 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:11,998 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:11,998 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,998 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,998 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:11,999 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:11,999 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:11,999 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:11,999 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:12,000 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:12,000 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b to restore RIT 2023-02-09 22:54:12,000 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983282000 2023-02-09 22:54:12,001 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:12,001 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,004 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:12,004 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSING, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:12,004 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:12,004 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983312004 2023-02-09 22:54:12,004 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983372004 2023-02-09 22:54:12,004 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:12,004 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:12,005 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983252004"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983252004"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983252004"}]},"ts":"1675983252004"} 2023-02-09 22:54:12,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:12,006 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:12,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing c0d992ad86816a23e23291a2fe7d092b, disabling compactions & flushes 2023-02-09 22:54:12,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:12,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:12,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. after waiting 0 ms 2023-02-09 22:54:12,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:12,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing c0d992ad86816a23e23291a2fe7d092b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 22:54:12,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/e3a386e4ac64493fb930cdb9d503daf9 2023-02-09 22:54:12,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/69ee19d61d204bdda4b13fa4f4715ba7 2023-02-09 22:54:12,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf1/e3a386e4ac64493fb930cdb9d503daf9 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9 2023-02-09 22:54:12,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:54:12,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/.tmp/cf2/69ee19d61d204bdda4b13fa4f4715ba7 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7 2023-02-09 22:54:12,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:54:12,215 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 55ms, sequenceid=82, compaction requested=true 2023-02-09 22:54:12,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 22:54:12,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:12,236 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. 2023-02-09 22:54:12,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for c0d992ad86816a23e23291a2fe7d092b: Waiting for close lock at 1675983252160Running coprocessor pre-close hooks at 1675983252160Disabling compacts and flushes for region at 1675983252160Disabling writes for close at 1675983252160Obtaining lock to block concurrent updates at 1675983252161 (+1 ms)Preparing flush snapshotting stores in c0d992ad86816a23e23291a2fe7d092b at 1675983252161Finished memstore snapshotting testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675983252161Flushing stores of testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. at 1675983252161Flushing c0d992ad86816a23e23291a2fe7d092b/cf1: creating writer at 1675983252162 (+1 ms)Flushing c0d992ad86816a23e23291a2fe7d092b/cf1: appending metadata at 1675983252165 (+3 ms)Flushing c0d992ad86816a23e23291a2fe7d092b/cf1: closing flushed file at 1675983252165Flushing c0d992ad86816a23e23291a2fe7d092b/cf2: creating writer at 1675983252179 (+14 ms)Flushing c0d992ad86816a23e23291a2fe7d092b/cf2: appending metadata at 1675983252181 (+2 ms)Flushing c0d992ad86816a23e23291a2fe7d092b/cf2: closing flushed file at 1675983252181Flushing c0d992ad86816a23e23291a2fe7d092b/cf1: reopening flushed file at 1675983252204 (+23 ms)Flushing c0d992ad86816a23e23291a2fe7d092b/cf2: reopening flushed file at 1675983252210 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c0d992ad86816a23e23291a2fe7d092b in 55ms, sequenceid=82, compaction requested=true at 1675983252215 (+5 ms)Writing region close event to WAL at 1675983252232 (+17 ms)Running coprocessor post-close hooks at 1675983252236 (+4 ms)Closed at 1675983252236 2023-02-09 22:54:12,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,239 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED 2023-02-09 22:54:12,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983252239"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983252239"}]},"ts":"1675983252239"} 2023-02-09 22:54:12,241 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:12,241 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:12,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:12,241 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:12,254 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,255 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:12,255 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:12,255 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:12,255 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:12,255 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:12,255 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:12,255 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:12,255 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:12,256 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:12,257 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:12,257 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 22:54:12,257 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 22:54:12,257 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:12,258 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:12,258 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:12,258 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:12,258 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,259 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:12,259 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,259 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,259 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:12,260 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,260 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,260 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:12,260 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:12,260 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,261 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,261 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,261 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:12,261 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,261 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:12,261 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:12,261 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:12,262 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:12,262 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b to restore RIT 2023-02-09 22:54:12,262 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983282262 2023-02-09 22:54:12,263 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:12,263 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:12,265 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:12,265 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:12,265 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:12,265 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983312265 2023-02-09 22:54:12,265 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983372265 2023-02-09 22:54:12,266 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 22:54:12,266 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:12,266 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983252266"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983252266"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983252266"}]},"ts":"1675983252266"} 2023-02-09 22:54:12,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:12,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-09 22:54:12,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 in 261 msec 2023-02-09 22:54:12,270 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:12,270 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:12,270 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:12,516 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,516 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:12,516 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:12,516 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:12,516 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:12,516 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:12,516 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:12,516 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:12,517 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:12,517 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:12,518 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:12,519 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 22:54:12,519 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:12,519 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:12,520 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:12,520 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:12,520 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:12,520 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,521 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:12,521 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,521 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,522 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:12,522 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,522 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,522 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:12,522 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:12,522 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,523 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,523 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,523 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:12,523 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:12,523 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:12,523 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:12,524 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:12,524 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b to restore RIT 2023-02-09 22:54:12,524 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983282524 2023-02-09 22:54:12,525 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,525 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:12,528 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:12,528 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:12,528 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:12,528 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983312528 2023-02-09 22:54:12,528 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983372528 2023-02-09 22:54:12,528 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 22:54:12,528 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,530 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:12,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-09 22:54:12,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN in 789 msec 2023-02-09 22:54:12,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:12,536 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:12,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:12,778 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,778 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:12,779 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:12,779 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:12,779 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:12,779 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:12,779 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:12,779 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:12,779 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:12,780 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:12,781 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:12,781 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:12,781 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:12,781 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:12,782 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:12,782 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:12,782 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:12,783 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,783 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:12,783 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,783 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:12,784 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:12,784 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,784 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:12,784 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:12,784 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:12,784 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:12,785 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:12,785 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:12,785 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:12,786 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:12,786 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983282786 2023-02-09 22:54:12,787 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:12,787 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,790 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:12,790 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:12,790 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:12,790 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983312790 2023-02-09 22:54:12,790 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983372790 2023-02-09 22:54:12,790 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:12,790 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,790 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,790 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:12,795 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:12,804 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=c0d992ad86816a23e23291a2fe7d092b, threads=14 2023-02-09 22:54:12,806 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,806 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,806 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,806 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,807 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,807 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,807 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,807 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,807 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,807 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,807 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,808 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,808 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,808 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,919 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,922 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,930 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,952 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,955 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,963 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,972 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,979 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,983 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,991 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,992 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,993 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,993 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,994 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:12,995 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region c0d992ad86816a23e23291a2fe7d092b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:54:13,032 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:13,033 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:13,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:13,040 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,041 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:13,041 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:13,041 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:13,042 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:13,042 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:13,042 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:13,042 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:13,042 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:13,042 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:13,043 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 22:54:13,044 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:13,044 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:13,044 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:13,044 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:13,045 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:13,045 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:13,045 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:13,046 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:13,046 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:13,047 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:13,047 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:13,048 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,048 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:13,048 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:13,048 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:13,049 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,049 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,049 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,049 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:13,049 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:13,049 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:54:13,050 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:13,050 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:13,051 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983283051 2023-02-09 22:54:13,052 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:13,052 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,054 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:13,054 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:13,054 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:13,055 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983313055 2023-02-09 22:54:13,055 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983373055 2023-02-09 22:54:13,055 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:13,064 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=c0d992ad86816a23e23291a2fe7d092b, threads=14 2023-02-09 22:54:13,065 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,065 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,065 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,066 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,066 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,066 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,066 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,066 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,066 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,066 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,067 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,067 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,067 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,067 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,205 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,234 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,234 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,239 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,241 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,253 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,294 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,299 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,307 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,319 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,319 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,323 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,698 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,715 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261 for region: c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:13,715 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region c0d992ad86816a23e23291a2fe7d092b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:54:13,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:13,759 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:54:13,761 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:54:13,761 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:13,761 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:13,761 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:13,805 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,805 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:13,805 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:13,805 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:13,806 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:13,806 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:13,806 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:13,806 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:13,806 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:13,807 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:13,808 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 22:54:13,808 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:13,808 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:13,808 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:13,808 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:13,809 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:13,809 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:13,809 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:13,809 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:13,809 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:13,810 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:13,810 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:13,811 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,811 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:13,811 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:13,811 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:13,811 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,811 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,811 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,811 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:13,811 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:13,811 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:13,812 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:13,812 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:13,813 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983283813 2023-02-09 22:54:13,813 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:13,814 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,816 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:13,816 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:13,816 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:13,817 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983313817 2023-02-09 22:54:13,817 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983373817 2023-02-09 22:54:13,817 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:13,819 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:13,821 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:13,821 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:13,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:13,821 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:14,016 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:14,067 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:14,067 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:14,067 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:14,067 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:14,068 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:14,068 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:14,068 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:14,068 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:14,068 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:14,069 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:14,070 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 22:54:14,070 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:14,071 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:14,071 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:14,071 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:14,072 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:14,072 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:14,072 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:14,073 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:14,073 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:14,073 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:14,074 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:14,074 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,075 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN 2023-02-09 22:54:14,075 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:14,075 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:14,075 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,075 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,075 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,075 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c0d992ad86816a23e23291a2fe7d092b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:14,075 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure c0d992ad86816a23e23291a2fe7d092b, server=jenkins-hbase9.apache.org,44627,1675983241473 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:14,076 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:54:14,076 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:14,077 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:14,077 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983284077 2023-02-09 22:54:14,079 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,079 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:14,081 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:14,082 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=c0d992ad86816a23e23291a2fe7d092b, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,44627,1675983241473, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473, openSeqNum=2 2023-02-09 22:54:14,082 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:14,082 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983314082 2023-02-09 22:54:14,082 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983374082 2023-02-09 22:54:14,082 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:14,084 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983254084"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675983254084"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675983254084"}]},"ts":"1675983254084"} 2023-02-09 22:54:14,084 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983254084"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983254084"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983254084"}]},"ts":"1675983254084"} 2023-02-09 22:54:14,085 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983254084"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983254084"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983254084"}]},"ts":"1675983254084"} 2023-02-09 22:54:14,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=80ed137b13594853114cbdbfcc0d5699, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c029d6f8a5817f8d8305ca00520f98be, ASSIGN}] 2023-02-09 22:54:14,091 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c029d6f8a5817f8d8305ca00520f98be, ASSIGN 2023-02-09 22:54:14,092 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c029d6f8a5817f8d8305ca00520f98be, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=false 2023-02-09 22:54:14,092 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=80ed137b13594853114cbdbfcc0d5699, ASSIGN 2023-02-09 22:54:14,093 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=80ed137b13594853114cbdbfcc0d5699, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=false 2023-02-09 22:54:14,242 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 22:54:14,244 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=c029d6f8a5817f8d8305ca00520f98be, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:14,244 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983254243"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983254243"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983254243"}]},"ts":"1675983254243"} 2023-02-09 22:54:14,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure c029d6f8a5817f8d8305ca00520f98be, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:14,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=80ed137b13594853114cbdbfcc0d5699, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:14,250 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983254249"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983254249"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983254249"}]},"ts":"1675983254249"} 2023-02-09 22:54:14,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 80ed137b13594853114cbdbfcc0d5699, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:14,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:14,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c029d6f8a5817f8d8305ca00520f98be, NAME => 'testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 22:54:14,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:14,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:14,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,403 INFO [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,404 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1 2023-02-09 22:54:14,404 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1 2023-02-09 22:54:14,405 INFO [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c029d6f8a5817f8d8305ca00520f98be columnFamilyName cf1 2023-02-09 22:54:14,419 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad-top 2023-02-09 22:54:14,423 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07-top 2023-02-09 22:54:14,427 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49-top 2023-02-09 22:54:14,432 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee-top 2023-02-09 22:54:14,436 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6-top 2023-02-09 22:54:14,440 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388-top 2023-02-09 22:54:14,444 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9-top 2023-02-09 22:54:14,444 INFO [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(310): Store=c029d6f8a5817f8d8305ca00520f98be/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:14,444 INFO [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,445 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2 2023-02-09 22:54:14,445 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2 2023-02-09 22:54:14,446 INFO [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c029d6f8a5817f8d8305ca00520f98be columnFamilyName cf2 2023-02-09 22:54:14,461 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98-top 2023-02-09 22:54:14,465 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7-top 2023-02-09 22:54:14,469 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c-top 2023-02-09 22:54:14,473 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff-top 2023-02-09 22:54:14,477 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057-top 2023-02-09 22:54:14,481 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261-top 2023-02-09 22:54:14,484 DEBUG [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521-top 2023-02-09 22:54:14,484 INFO [StoreOpener-c029d6f8a5817f8d8305ca00520f98be-1] regionserver.HStore(310): Store=c029d6f8a5817f8d8305ca00520f98be/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:14,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:14,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:14,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened c029d6f8a5817f8d8305ca00520f98be; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74644199, jitterRate=0.11228524148464203}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:14,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for c029d6f8a5817f8d8305ca00520f98be: Running coprocessor pre-open hook at 1675983254402Writing region info on filesystem at 1675983254402Initializing all the Stores at 1675983254403 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983254403Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983254403Cleaning up temporary data from old regions at 1675983254487 (+84 ms)Cleaning up detritus from prior splits at 1675983254487Running coprocessor post-open hooks at 1675983254490 (+3 ms)Region opened successfully at 1675983254490 2023-02-09 22:54:14,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be., pid=33, masterSystemTime=1675983254398 2023-02-09 22:54:14,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:14,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:14,492 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:14,492 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:14,492 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:14,492 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1912): c029d6f8a5817f8d8305ca00520f98be/cf2 is initiating minor compaction (all files) 2023-02-09 22:54:14,492 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:14,492 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c029d6f8a5817f8d8305ca00520f98be/cf2 in testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:14,492 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1912): c029d6f8a5817f8d8305ca00520f98be/cf1 is initiating minor compaction (all files) 2023-02-09 22:54:14,492 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c029d6f8a5817f8d8305ca00520f98be/cf1 in testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:14,492 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7-top] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/.tmp, totalSize=34.9 K 2023-02-09 22:54:14,493 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9-top] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/.tmp, totalSize=34.9 K 2023-02-09 22:54:14,493 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983249452 2023-02-09 22:54:14,493 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983249452 2023-02-09 22:54:14,494 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983249648 2023-02-09 22:54:14,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:14,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:14,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:14,494 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983249648 2023-02-09 22:54:14,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 80ed137b13594853114cbdbfcc0d5699, NAME => 'testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 22:54:14,494 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983249855 2023-02-09 22:54:14,494 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=c029d6f8a5817f8d8305ca00520f98be, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:14,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:14,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983254494"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983254494"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983254494"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983254494"}]},"ts":"1675983254494"} 2023-02-09 22:54:14,495 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983249855 2023-02-09 22:54:14,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,495 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983250056 2023-02-09 22:54:14,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:14,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,495 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983250056 2023-02-09 22:54:14,495 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675983250254 2023-02-09 22:54:14,496 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675983250254 2023-02-09 22:54:14,496 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675983250464 2023-02-09 22:54:14,496 INFO [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,496 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675983250464 2023-02-09 22:54:14,496 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983250672 2023-02-09 22:54:14,497 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983250672 2023-02-09 22:54:14,498 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1 2023-02-09 22:54:14,498 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1 2023-02-09 22:54:14,499 INFO [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80ed137b13594853114cbdbfcc0d5699 columnFamilyName cf1 2023-02-09 22:54:14,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-09 22:54:14,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure c029d6f8a5817f8d8305ca00520f98be, server=jenkins-hbase9.apache.org,44627,1675983241473 in 254 msec 2023-02-09 22:54:14,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c029d6f8a5817f8d8305ca00520f98be, ASSIGN in 415 msec 2023-02-09 22:54:14,520 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] throttle.PressureAwareThroughputController(145): c029d6f8a5817f8d8305ca00520f98be#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:14,529 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad-bottom 2023-02-09 22:54:14,532 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] throttle.PressureAwareThroughputController(145): c029d6f8a5817f8d8305ca00520f98be#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:14,540 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07-bottom 2023-02-09 22:54:14,545 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49-bottom 2023-02-09 22:54:14,549 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee-bottom 2023-02-09 22:54:14,552 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6-bottom 2023-02-09 22:54:14,557 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388-bottom 2023-02-09 22:54:14,567 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9-bottom 2023-02-09 22:54:14,567 INFO [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(310): Store=80ed137b13594853114cbdbfcc0d5699/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:14,567 INFO [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,568 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2 2023-02-09 22:54:14,568 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/.tmp/cf1/a0acaf10b1e24ae0b6b3ad7089868547 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/a0acaf10b1e24ae0b6b3ad7089868547 2023-02-09 22:54:14,568 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2 2023-02-09 22:54:14,569 INFO [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80ed137b13594853114cbdbfcc0d5699 columnFamilyName cf2 2023-02-09 22:54:14,571 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/.tmp/cf2/eb6f39fa23644b3e96ba42c2b57bfea2 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/eb6f39fa23644b3e96ba42c2b57bfea2 2023-02-09 22:54:14,575 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c029d6f8a5817f8d8305ca00520f98be/cf1 of c029d6f8a5817f8d8305ca00520f98be into a0acaf10b1e24ae0b6b3ad7089868547(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:14,575 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c029d6f8a5817f8d8305ca00520f98be: 2023-02-09 22:54:14,575 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be., storeName=c029d6f8a5817f8d8305ca00520f98be/cf1, priority=9, startTime=1675983254491; duration=0sec 2023-02-09 22:54:14,575 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:14,577 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c029d6f8a5817f8d8305ca00520f98be/cf2 of c029d6f8a5817f8d8305ca00520f98be into eb6f39fa23644b3e96ba42c2b57bfea2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:14,577 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c029d6f8a5817f8d8305ca00520f98be: 2023-02-09 22:54:14,577 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be., storeName=c029d6f8a5817f8d8305ca00520f98be/cf2, priority=9, startTime=1675983254491; duration=0sec 2023-02-09 22:54:14,577 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:14,584 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98-bottom 2023-02-09 22:54:14,588 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7-bottom 2023-02-09 22:54:14,591 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c-bottom 2023-02-09 22:54:14,594 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff-bottom 2023-02-09 22:54:14,598 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057-bottom 2023-02-09 22:54:14,602 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261-bottom 2023-02-09 22:54:14,606 DEBUG [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521-bottom 2023-02-09 22:54:14,606 INFO [StoreOpener-80ed137b13594853114cbdbfcc0d5699-1] regionserver.HStore(310): Store=80ed137b13594853114cbdbfcc0d5699/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:14,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:14,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:14,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 80ed137b13594853114cbdbfcc0d5699; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72358673, jitterRate=0.07822825014591217}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:14,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 80ed137b13594853114cbdbfcc0d5699: Running coprocessor pre-open hook at 1675983254495Writing region info on filesystem at 1675983254495Initializing all the Stores at 1675983254496 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983254496Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983254496Cleaning up temporary data from old regions at 1675983254608 (+112 ms)Cleaning up detritus from prior splits at 1675983254609 (+1 ms)Running coprocessor post-open hooks at 1675983254611 (+2 ms)Region opened successfully at 1675983254611 2023-02-09 22:54:14,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699., pid=34, masterSystemTime=1675983254398 2023-02-09 22:54:14,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:14,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:14,613 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:14,613 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:14,613 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:14,613 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:14,613 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1912): 80ed137b13594853114cbdbfcc0d5699/cf1 is initiating minor compaction (all files) 2023-02-09 22:54:14,613 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1912): 80ed137b13594853114cbdbfcc0d5699/cf2 is initiating minor compaction (all files) 2023-02-09 22:54:14,613 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 80ed137b13594853114cbdbfcc0d5699/cf1 in testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:14,614 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 80ed137b13594853114cbdbfcc0d5699/cf2 in testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:14,614 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/.tmp, totalSize=34.9 K 2023-02-09 22:54:14,614 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/.tmp, totalSize=34.9 K 2023-02-09 22:54:14,614 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983249452 2023-02-09 22:54:14,615 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983249452 2023-02-09 22:54:14,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:14,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:14,615 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983249648 2023-02-09 22:54:14,615 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983249648 2023-02-09 22:54:14,616 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983249855 2023-02-09 22:54:14,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=80ed137b13594853114cbdbfcc0d5699, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:14,616 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983249855 2023-02-09 22:54:14,616 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983254616"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983254616"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983254616"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983254616"}]},"ts":"1675983254616"} 2023-02-09 22:54:14,616 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983250056 2023-02-09 22:54:14,616 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983250056 2023-02-09 22:54:14,616 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983250254 2023-02-09 22:54:14,617 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983250254 2023-02-09 22:54:14,617 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983250464 2023-02-09 22:54:14,617 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983250464 2023-02-09 22:54:14,618 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983250672 2023-02-09 22:54:14,618 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983250672 2023-02-09 22:54:14,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-09 22:54:14,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 80ed137b13594853114cbdbfcc0d5699, server=jenkins-hbase9.apache.org,44627,1675983241473 in 367 msec 2023-02-09 22:54:14,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-09 22:54:14,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=80ed137b13594853114cbdbfcc0d5699, ASSIGN in 533 msec 2023-02-09 22:54:14,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be in 3.6880 sec 2023-02-09 22:54:14,633 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] throttle.PressureAwareThroughputController(145): 80ed137b13594853114cbdbfcc0d5699#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:14,635 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 80ed137b13594853114cbdbfcc0d5699#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:14,649 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/.tmp/cf2/d0bc4fed9ea042beb87e4c2a26906e5e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/d0bc4fed9ea042beb87e4c2a26906e5e 2023-02-09 22:54:14,650 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/.tmp/cf1/d2905f9cac4c44e798fae17ad305fc91 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/d2905f9cac4c44e798fae17ad305fc91 2023-02-09 22:54:14,655 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 80ed137b13594853114cbdbfcc0d5699/cf2 of 80ed137b13594853114cbdbfcc0d5699 into d0bc4fed9ea042beb87e4c2a26906e5e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:14,655 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 80ed137b13594853114cbdbfcc0d5699/cf1 of 80ed137b13594853114cbdbfcc0d5699 into d2905f9cac4c44e798fae17ad305fc91(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:14,655 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 80ed137b13594853114cbdbfcc0d5699: 2023-02-09 22:54:14,655 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 80ed137b13594853114cbdbfcc0d5699: 2023-02-09 22:54:14,655 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699., storeName=80ed137b13594853114cbdbfcc0d5699/cf2, priority=9, startTime=1675983254613; duration=0sec 2023-02-09 22:54:14,656 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699., storeName=80ed137b13594853114cbdbfcc0d5699/cf1, priority=9, startTime=1675983254612; duration=0sec 2023-02-09 22:54:14,656 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:14,656 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:14,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44627] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.2.10:44710 deadline: 1675983314833, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b. is not online on jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:14,945 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 22:54:14,953 INFO [Listener at localhost/44743] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 22:54:14,956 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 22:54:14,986 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:14,988 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-09 22:54:14,989 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testSplitWithoutPONR 2023-02-09 22:54:14,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:14,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:14,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983254993"}]},"ts":"1675983254993"} 2023-02-09 22:54:14,995 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-09 22:54:14,997 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-09 22:54:14,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=80ed137b13594853114cbdbfcc0d5699, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c029d6f8a5817f8d8305ca00520f98be, UNASSIGN}] 2023-02-09 22:54:14,998 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c029d6f8a5817f8d8305ca00520f98be, UNASSIGN 2023-02-09 22:54:14,999 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=c029d6f8a5817f8d8305ca00520f98be, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:14,999 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983254999"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983254999"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983254999"}]},"ts":"1675983254999"} 2023-02-09 22:54:15,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure c029d6f8a5817f8d8305ca00520f98be, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:15,004 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=80ed137b13594853114cbdbfcc0d5699, UNASSIGN 2023-02-09 22:54:15,004 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=80ed137b13594853114cbdbfcc0d5699, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:15,005 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983255004"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983255004"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983255004"}]},"ts":"1675983255004"} 2023-02-09 22:54:15,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 80ed137b13594853114cbdbfcc0d5699, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:15,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:15,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:15,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 80ed137b13594853114cbdbfcc0d5699, disabling compactions & flushes 2023-02-09 22:54:15,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:15,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:15,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. after waiting 0 ms 2023-02-09 22:54:15,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:15,165 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9-bottom] to archive 2023-02-09 22:54:15,166 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:15,167 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,169 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,170 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,171 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,172 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,174 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,175 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,177 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7-bottom] to archive 2023-02-09 22:54:15,178 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:15,179 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,180 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,182 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,183 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,184 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,185 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,186 DEBUG [StoreCloser-testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:54:15,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:15,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699. 2023-02-09 22:54:15,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 80ed137b13594853114cbdbfcc0d5699: Waiting for close lock at 1675983255152Running coprocessor pre-close hooks at 1675983255152Disabling compacts and flushes for region at 1675983255152Disabling writes for close at 1675983255153 (+1 ms)Writing region close event to WAL at 1675983255187 (+34 ms)Running coprocessor post-close hooks at 1675983255191 (+4 ms)Closed at 1675983255191 2023-02-09 22:54:15,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:15,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:15,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing c029d6f8a5817f8d8305ca00520f98be, disabling compactions & flushes 2023-02-09 22:54:15,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:15,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:15,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. after waiting 0 ms 2023-02-09 22:54:15,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:15,193 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=80ed137b13594853114cbdbfcc0d5699, regionState=CLOSED 2023-02-09 22:54:15,193 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983255193"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983255193"}]},"ts":"1675983255193"} 2023-02-09 22:54:15,194 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9-top] to archive 2023-02-09 22:54:15,195 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:15,197 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-09 22:54:15,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 80ed137b13594853114cbdbfcc0d5699, server=jenkins-hbase9.apache.org,44627,1675983241473 in 189 msec 2023-02-09 22:54:15,198 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=80ed137b13594853114cbdbfcc0d5699, UNASSIGN in 200 msec 2023-02-09 22:54:15,200 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,201 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,202 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,203 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,205 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,206 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7-top] to archive 2023-02-09 22:54:15,207 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:15,209 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,210 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,211 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,212 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,213 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,214 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,215 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:54:15,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:15,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be. 2023-02-09 22:54:15,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for c029d6f8a5817f8d8305ca00520f98be: Waiting for close lock at 1675983255193Running coprocessor pre-close hooks at 1675983255193Disabling compacts and flushes for region at 1675983255193Disabling writes for close at 1675983255193Writing region close event to WAL at 1675983255216 (+23 ms)Running coprocessor post-close hooks at 1675983255220 (+4 ms)Closed at 1675983255220 2023-02-09 22:54:15,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:15,222 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=c029d6f8a5817f8d8305ca00520f98be, regionState=CLOSED 2023-02-09 22:54:15,222 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983255222"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983255222"}]},"ts":"1675983255222"} 2023-02-09 22:54:15,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-09 22:54:15,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure c029d6f8a5817f8d8305ca00520f98be, server=jenkins-hbase9.apache.org,44627,1675983241473 in 223 msec 2023-02-09 22:54:15,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-09 22:54:15,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c029d6f8a5817f8d8305ca00520f98be, UNASSIGN in 228 msec 2023-02-09 22:54:15,229 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983255229"}]},"ts":"1675983255229"} 2023-02-09 22:54:15,231 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-09 22:54:15,232 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-09 22:54:15,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 244 msec 2023-02-09 22:54:15,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:15,296 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-09 22:54:15,297 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testSplitWithoutPONR 2023-02-09 22:54:15,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:15,299 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:15,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:15,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 22:54:15,306 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,306 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:15,306 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:15,309 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/recovered.edits] 2023-02-09 22:54:15,309 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/recovered.edits] 2023-02-09 22:54:15,310 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/recovered.edits] 2023-02-09 22:54:15,314 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/.regioninfo to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/.regioninfo 2023-02-09 22:54:15,314 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/231ae60a162048d6a3f4b7608da116ad 2023-02-09 22:54:15,315 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/.regioninfo to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/.regioninfo 2023-02-09 22:54:15,316 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/2f52cfa283cf47b99ae5c9885682ea07 2023-02-09 22:54:15,318 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/40236328589d4c58af264fba8cd36b49 2023-02-09 22:54:15,319 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,319 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/231ae60a162048d6a3f4b7608da116ad.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,320 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/472f8708d9c54b3bb24b0c944d5df1ee 2023-02-09 22:54:15,320 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,321 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/2f52cfa283cf47b99ae5c9885682ea07.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,322 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,322 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/4f9a534fdf5e4d5e981d6dff142702a6 2023-02-09 22:54:15,322 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/40236328589d4c58af264fba8cd36b49.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,323 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,324 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/ce7e0ba501254a2ba773d9afe11b3388 2023-02-09 22:54:15,324 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/472f8708d9c54b3bb24b0c944d5df1ee.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,325 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,325 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf1/e3a386e4ac64493fb930cdb9d503daf9 2023-02-09 22:54:15,326 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/4f9a534fdf5e4d5e981d6dff142702a6.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,327 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,329 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/ce7e0ba501254a2ba773d9afe11b3388.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,329 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,330 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/1e80ec5a4cf04a09ae932450776f1f98 2023-02-09 22:54:15,330 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf1/e3a386e4ac64493fb930cdb9d503daf9.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,332 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/69ee19d61d204bdda4b13fa4f4715ba7 2023-02-09 22:54:15,333 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,334 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/7c934e776f20418c98676a7d5c97f74c 2023-02-09 22:54:15,340 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/1e80ec5a4cf04a09ae932450776f1f98.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,340 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,341 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/bb901fac0d0048d58bd40d9ae7b819ff 2023-02-09 22:54:15,342 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/69ee19d61d204bdda4b13fa4f4715ba7.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,343 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,343 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/e30f2d55e86e4fbb9a051c52fbb6f057 2023-02-09 22:54:15,345 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/7c934e776f20418c98676a7d5c97f74c.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,345 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,345 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/f1235bdef7f0484ebaabf1b9f790d261 2023-02-09 22:54:15,347 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/bb901fac0d0048d58bd40d9ae7b819ff.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,347 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/cf2/fd613ae8690648efa8637dbb144b8521 2023-02-09 22:54:15,347 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,348 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/e30f2d55e86e4fbb9a051c52fbb6f057.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,349 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,350 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/f1235bdef7f0484ebaabf1b9f790d261.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,351 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/recovered.edits/85.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b/recovered.edits/85.seqid 2023-02-09 22:54:15,351 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/c029d6f8a5817f8d8305ca00520f98be/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,352 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,353 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/80ed137b13594853114cbdbfcc0d5699/cf2/fd613ae8690648efa8637dbb144b8521.c0d992ad86816a23e23291a2fe7d092b 2023-02-09 22:54:15,355 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/a0acaf10b1e24ae0b6b3ad7089868547 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf1/a0acaf10b1e24ae0b6b3ad7089868547 2023-02-09 22:54:15,356 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/d2905f9cac4c44e798fae17ad305fc91 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf1/d2905f9cac4c44e798fae17ad305fc91 2023-02-09 22:54:15,358 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/eb6f39fa23644b3e96ba42c2b57bfea2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/cf2/eb6f39fa23644b3e96ba42c2b57bfea2 2023-02-09 22:54:15,359 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/d0bc4fed9ea042beb87e4c2a26906e5e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/cf2/d0bc4fed9ea042beb87e4c2a26906e5e 2023-02-09 22:54:15,361 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be/recovered.edits/90.seqid 2023-02-09 22:54:15,363 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:15,363 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699/recovered.edits/90.seqid 2023-02-09 22:54:15,364 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitWithoutPONR/80ed137b13594853114cbdbfcc0d5699 2023-02-09 22:54:15,364 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 22:54:15,367 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:15,370 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-09 22:54:15,372 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-09 22:54:15,374 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:15,374 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-09 22:54:15,374 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983255374"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:15,374 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983255374"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:15,374 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983255374"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:15,376 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:54:15,376 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c0d992ad86816a23e23291a2fe7d092b, NAME => 'testSplitWithoutPONR,,1675983248947.c0d992ad86816a23e23291a2fe7d092b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 80ed137b13594853114cbdbfcc0d5699, NAME => 'testSplitWithoutPONR,,1675983250935.80ed137b13594853114cbdbfcc0d5699.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => c029d6f8a5817f8d8305ca00520f98be, NAME => 'testSplitWithoutPONR,41,1675983250935.c029d6f8a5817f8d8305ca00520f98be.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 22:54:15,376 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-09 22:54:15,376 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983255376"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:15,378 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-09 22:54:15,381 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:15,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 84 msec 2023-02-09 22:54:15,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 22:54:15,402 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-09 22:54:15,418 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=403 (was 395) Potentially hanging thread: RS:1;jenkins-hbase9:44627-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:44626 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:44870 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:36794 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:45950 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:45466 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:36782 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:59014 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xaab1b33-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:58470 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:44906 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:36616 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:40396 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:36810 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:36776 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:44922 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:40384 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1301771038_15 at /127.0.0.1:40380 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=856 (was 849) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=222 (was 210) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=11033 (was 11188) 2023-02-09 22:54:15,434 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=403, OpenFileDescriptor=856, MaxFileDescriptor=60000, SystemLoadAverage=222, ProcessCount=168, AvailableMemoryMB=11032 2023-02-09 22:54:15,434 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:15,437 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:54:15,437 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:54:15,439 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:15,441 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:15,442 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:15,443 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,444 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd empty. 2023-02-09 22:54:15,445 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 22:54:15,457 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:15,458 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => f0f398cb96c5ad5c92cff9427e7b68fd, NAME => 'testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:54:15,470 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:15,470 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing f0f398cb96c5ad5c92cff9427e7b68fd, disabling compactions & flushes 2023-02-09 22:54:15,470 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,470 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,470 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. after waiting 0 ms 2023-02-09 22:54:15,470 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,470 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,470 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for f0f398cb96c5ad5c92cff9427e7b68fd: Waiting for close lock at 1675983255470Disabling compacts and flushes for region at 1675983255470Disabling writes for close at 1675983255470Writing region close event to WAL at 1675983255470Closed at 1675983255470 2023-02-09 22:54:15,472 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:15,473 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983255473"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983255473"}]},"ts":"1675983255473"} 2023-02-09 22:54:15,475 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:15,476 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:15,476 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983255476"}]},"ts":"1675983255476"} 2023-02-09 22:54:15,477 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 22:54:15,480 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:15,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:15,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:15,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:15,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:15,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN}] 2023-02-09 22:54:15,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:15,484 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41885,1675983241428; forceNewPlan=false, retain=false 2023-02-09 22:54:15,634 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:15,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:15,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983255636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983255636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983255636"}]},"ts":"1675983255636"} 2023-02-09 22:54:15,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:15,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f0f398cb96c5ad5c92cff9427e7b68fd, NAME => 'testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:15,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:15,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:15,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,802 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,803 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1 2023-02-09 22:54:15,803 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1 2023-02-09 22:54:15,804 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0f398cb96c5ad5c92cff9427e7b68fd columnFamilyName cf1 2023-02-09 22:54:15,804 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(310): Store=f0f398cb96c5ad5c92cff9427e7b68fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:15,804 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,806 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2 2023-02-09 22:54:15,806 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2 2023-02-09 22:54:15,806 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0f398cb96c5ad5c92cff9427e7b68fd columnFamilyName cf2 2023-02-09 22:54:15,807 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(310): Store=f0f398cb96c5ad5c92cff9427e7b68fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:15,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:15,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:15,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:15,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened f0f398cb96c5ad5c92cff9427e7b68fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63164778, jitterRate=-0.05877146124839783}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:15,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for f0f398cb96c5ad5c92cff9427e7b68fd: Running coprocessor pre-open hook at 1675983255801Writing region info on filesystem at 1675983255801Initializing all the Stores at 1675983255801Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983255801Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983255801Cleaning up temporary data from old regions at 1675983255809 (+8 ms)Cleaning up detritus from prior splits at 1675983255809Running coprocessor post-open hooks at 1675983255813 (+4 ms)Region opened successfully at 1675983255813 2023-02-09 22:54:15,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd., pid=43, masterSystemTime=1675983255791 2023-02-09 22:54:15,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,817 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:15,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983255816"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983255816"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983255816"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983255816"}]},"ts":"1675983255816"} 2023-02-09 22:54:15,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-09 22:54:15,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 in 179 msec 2023-02-09 22:54:15,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-09 22:54:15,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN in 339 msec 2023-02-09 22:54:15,823 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:15,823 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983255823"}]},"ts":"1675983255823"} 2023-02-09 22:54:15,824 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 22:54:15,830 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:15,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 392 msec 2023-02-09 22:54:15,942 DEBUG [Listener at localhost/44743] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:15,945 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:37162, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:15,953 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:15,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 22:54:15,955 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 22:54:15,955 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:15,955 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 22:54:15,955 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:54:15,956 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:15,956 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:15,957 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:15,957 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:15,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:15,957 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:15,957 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:15,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:15,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:15,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:15,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:15,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:15,958 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:15,958 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:15,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:15,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:15,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:15,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:15,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:15,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:15,959 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:15,959 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:15,959 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:15,959 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:15,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:15,959 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:15,960 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:15,960 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:15,960 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. started... 2023-02-09 22:54:15,961 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0f398cb96c5ad5c92cff9427e7b68fd 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:54:15,961 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:15,963 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:15,963 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:15,963 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:15,963 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:15,963 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:15,973 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/5443cf4e699145198d0cf0313fec8a79 2023-02-09 22:54:15,987 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/96b3675d98f942068ef1101e13fef51e 2023-02-09 22:54:15,992 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/5443cf4e699145198d0cf0313fec8a79 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 2023-02-09 22:54:15,996 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:15,996 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/96b3675d98f942068ef1101e13fef51e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e 2023-02-09 22:54:16,001 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:16,001 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 40ms, sequenceid=6, compaction requested=false 2023-02-09 22:54:16,002 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,002 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:16,002 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,002 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:16,002 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:16,002 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,002 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,002 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,005 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,005 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,006 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,006 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 22:54:16,008 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:54:16,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ee1fb32[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:16,008 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,009 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,009 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,010 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:16,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:16,010 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,012 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,012 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:54:16,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:54:16,018 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 22:54:16,018 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:16,018 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:16,019 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 22:54:16,019 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:16,020 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,020 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,020 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,020 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,020 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,021 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,021 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,021 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,022 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,022 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,024 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,024 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,025 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,029 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,030 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,029 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,029 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,030 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:16,030 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,030 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,030 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:16,031 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,031 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,031 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,031 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,031 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,031 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,131 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:16,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:16,158 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:16,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:16,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 22:54:16,160 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,160 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,160 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 22:54:16,160 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:54:16,161 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,161 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,163 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,163 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,163 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,163 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,164 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,164 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,175 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,175 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,175 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,175 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,176 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,175 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,176 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,176 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,176 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,175 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,176 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,176 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,176 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,178 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,179 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:16,179 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:16,179 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,179 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,179 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. started... 2023-02-09 22:54:16,179 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0f398cb96c5ad5c92cff9427e7b68fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:16,180 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,180 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,181 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,182 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,204 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/97e3ebd578dd48f69ef0a16fddf97617 2023-02-09 22:54:16,249 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/e49f24bfca344b5b856595fa4f952f5d 2023-02-09 22:54:16,255 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/97e3ebd578dd48f69ef0a16fddf97617 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 2023-02-09 22:54:16,264 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:16,265 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/e49f24bfca344b5b856595fa4f952f5d as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d 2023-02-09 22:54:16,271 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:16,272 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 93ms, sequenceid=19, compaction requested=false 2023-02-09 22:54:16,272 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:16,272 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,272 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:16,272 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:16,272 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,273 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,273 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,276 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,277 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,278 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,278 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 22:54:16,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58a9c39f[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:16,280 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:54:16,281 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,282 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,282 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:16,282 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,282 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,283 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,282 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,283 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:16,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,283 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:16,283 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,283 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,283 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:16,284 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,284 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,284 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,285 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,295 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,295 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,295 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,295 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,296 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,296 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:54:16,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:54:16,298 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 22:54:16,299 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:16,299 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:16,299 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 22:54:16,299 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:16,301 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,302 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,302 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,302 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,302 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,303 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,304 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,305 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,305 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,308 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,309 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:16,308 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,308 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:16,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,309 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,310 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,310 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:16,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,310 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,410 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:16,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:16,425 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:16,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:16,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 22:54:16,427 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,427 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,427 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 22:54:16,427 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:54:16,428 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,428 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,430 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,430 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,430 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:16,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,432 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,432 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,432 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:16,432 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. started... 2023-02-09 22:54:16,432 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,433 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,433 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,433 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0f398cb96c5ad5c92cff9427e7b68fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:16,434 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,435 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,435 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,435 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,435 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,435 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,442 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/5111ff6c32bd449482e6e2900efdfff5 2023-02-09 22:54:16,454 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/90dd446abaed402ba45bf28f11399baf 2023-02-09 22:54:16,459 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/5111ff6c32bd449482e6e2900efdfff5 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 2023-02-09 22:54:16,463 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:16,463 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/90dd446abaed402ba45bf28f11399baf as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf 2023-02-09 22:54:16,467 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:16,468 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 35ms, sequenceid=32, compaction requested=true 2023-02-09 22:54:16,468 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:16,468 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,468 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:16,468 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:16,468 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,469 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,469 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,470 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,470 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,470 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,471 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 22:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62f16e2e[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:16,472 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:54:16,472 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,475 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:16,475 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,475 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,476 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,476 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:16,476 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,476 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:16,476 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:16,477 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,477 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,477 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,478 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,478 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,478 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,479 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,479 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,479 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,479 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:54:16,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:54:16,483 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 22:54:16,483 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:16,483 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:16,483 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 22:54:16,483 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:16,485 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,485 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,485 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,485 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,485 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,485 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,486 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,490 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,490 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,490 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,498 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,498 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,498 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,498 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,498 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,498 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,498 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,499 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,498 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,499 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:16,498 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,499 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,499 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,499 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,499 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:16,499 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:16,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,499 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,500 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,500 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,500 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,500 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,500 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,600 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:16,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:16,615 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:16,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:16,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 22:54:16,616 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,616 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,616 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 22:54:16,617 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:54:16,617 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,617 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,618 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,618 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,618 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,618 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,619 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,619 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:16,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,620 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,621 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:16,621 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:16,621 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,621 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. started... 2023-02-09 22:54:16,621 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0f398cb96c5ad5c92cff9427e7b68fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:16,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,623 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,623 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,624 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,624 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,630 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/1b4348cf3799482c897a1e8fd618c04f 2023-02-09 22:54:16,642 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/938e4a6cfed349deba92dfbeef1c6ac3 2023-02-09 22:54:16,646 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/1b4348cf3799482c897a1e8fd618c04f as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f 2023-02-09 22:54:16,651 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:16,652 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/938e4a6cfed349deba92dfbeef1c6ac3 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 2023-02-09 22:54:16,657 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:16,658 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 37ms, sequenceid=45, compaction requested=true 2023-02-09 22:54:16,658 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:16,658 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,658 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:16,658 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:16,658 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,658 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,658 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,660 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,660 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,660 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,660 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 22:54:16,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@697b24f3[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:16,662 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:54:16,662 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:16,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,663 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:16,664 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:16,665 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,665 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,665 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,667 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,667 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,667 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,667 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,667 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,667 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,667 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:54:16,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:54:16,670 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 22:54:16,670 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:16,670 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:16,670 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 22:54:16,670 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:16,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,672 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,673 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,673 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,675 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,675 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,676 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,678 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,678 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,678 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,679 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,679 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,679 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:16,678 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:16,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:16,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,680 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,680 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,680 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,680 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,680 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,780 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:16,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:16,796 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:16,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:16,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 22:54:16,797 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,797 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,797 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 22:54:16,797 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:54:16,798 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,798 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,799 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,800 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,801 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,801 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:16,801 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. started... 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,801 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:16,802 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0f398cb96c5ad5c92cff9427e7b68fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:16,804 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,804 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,804 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,805 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,805 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,805 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,811 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/66bf557d13234f70a611073d685cef29 2023-02-09 22:54:16,823 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/dec37788d2ff4ce9a9247787f2e8a650 2023-02-09 22:54:16,827 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/66bf557d13234f70a611073d685cef29 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 2023-02-09 22:54:16,831 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:16,832 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/dec37788d2ff4ce9a9247787f2e8a650 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 2023-02-09 22:54:16,835 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:16,836 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 35ms, sequenceid=58, compaction requested=true 2023-02-09 22:54:16,836 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:16,836 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,836 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:16,836 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:16,836 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,836 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,836 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,838 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,838 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,838 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,838 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 22:54:16,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38127a85[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:16,840 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:54:16,840 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,841 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,841 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,841 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,841 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,841 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,841 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,842 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:16,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,841 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:16,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,842 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,842 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:16,842 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,842 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:16,842 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:16,842 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:16,844 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,844 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,844 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,847 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,847 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:16,847 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,847 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,847 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,847 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,848 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:16,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:54:16,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:54:16,850 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 22:54:16,850 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:16,850 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:16,850 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 22:54:16,850 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:16,852 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:16,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:16,852 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:16,852 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,852 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,853 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,853 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:16,855 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:16,855 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,856 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:16,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,858 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,860 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,860 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,861 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:16,861 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,861 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:16,861 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,861 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:16,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,861 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:16,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,861 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,861 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:16,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:16,861 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:16,861 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:16,961 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:16,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:16,977 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:16,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:16,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 22:54:16,980 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 22:54:16,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 22:54:16,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:54:16,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,983 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,983 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,983 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,983 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:16,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:16,984 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:16,984 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,985 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,985 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 22:54:16,985 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,985 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,985 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,985 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:16,986 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:16,986 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:16,986 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. started... 2023-02-09 22:54:16,986 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0f398cb96c5ad5c92cff9427e7b68fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:16,987 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,988 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,988 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,988 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:16,988 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:16,989 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:17,001 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/8b7790ab4c894920af68a72e08b6fa8c 2023-02-09 22:54:17,014 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/acf1c408ee5b4cf5bd529bebb06bd683 2023-02-09 22:54:17,018 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/8b7790ab4c894920af68a72e08b6fa8c as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c 2023-02-09 22:54:17,022 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:17,023 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/acf1c408ee5b4cf5bd529bebb06bd683 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 2023-02-09 22:54:17,028 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:17,029 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 43ms, sequenceid=71, compaction requested=true 2023-02-09 22:54:17,029 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:17,029 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:17,029 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:17,029 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:17,029 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,029 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 22:54:17,029 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 22:54:17,031 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,031 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:17,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:17,032 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,032 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:17,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:17,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:17,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:17,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 22:54:17,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2eb1f323[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:17,034 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:54:17,034 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:17,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,036 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:17,036 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:17,036 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:17,036 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:17,036 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:17,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,036 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:17,037 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:54:17,037 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 22:54:17,037 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:17,038 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:17,038 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:17,038 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:17,039 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,039 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:17,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:17,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:17,039 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:17,039 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 22:54:17,039 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:17,039 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:17,040 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 22:54:17,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:17,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:17,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:17,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:17,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:54:17,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:54:17,042 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 22:54:17,042 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:17,042 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:17,043 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 22:54:17,043 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:17,044 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,044 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:17,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:17,044 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,044 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:17,045 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:17,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,044 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:17,045 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:17,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,045 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:17,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:17,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,046 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:17,046 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:17,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 22:54:17,048 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,048 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,048 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:17,055 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:17,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:17,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:17,055 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 22:54:17,056 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:17,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:17,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:17,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:17,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:17,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:17,056 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:17,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,056 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:17,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:17,056 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 22:54:17,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:17,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:17,056 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:17,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:17,056 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,057 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,057 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:17,057 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 22:54:17,057 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 22:54:17,156 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:17,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:17,417 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 22:54:17,418 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:17,420 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,421 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,421 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,421 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:17,422 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:17,422 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:17,670 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,670 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:17,670 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:17,670 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:17,671 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-09 22:54:17,671 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:17,671 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:17,671 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:17,671 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:17,672 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:17,673 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 22:54:17,673 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:17,674 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:17,674 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:17,674 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:17,674 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:17,675 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:17,675 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:17,675 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:17,675 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:17,676 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:17,676 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:17,676 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:17,676 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:17,677 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,677 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:17,677 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,677 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,677 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,677 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:17,678 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:17,678 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:17,678 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983287678 2023-02-09 22:54:17,678 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:17,680 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,682 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:17,682 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:17,682 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:17,682 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983317682 2023-02-09 22:54:17,682 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983377682 2023-02-09 22:54:17,682 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:54:17,683 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:17,684 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:17,684 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:17,684 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:17,933 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,933 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:17,933 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:17,933 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:17,934 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:17,934 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:17,934 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:17,934 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:17,934 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:17,935 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:17,936 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 22:54:17,936 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:17,937 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:17,937 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:17,937 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:17,937 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:17,937 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:17,938 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:17,938 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:17,938 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:17,938 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:17,938 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:17,939 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:17,939 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:17,939 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,939 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:17,939 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,939 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,939 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,939 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:17,940 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:17,941 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:17,941 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983287941 2023-02-09 22:54:17,942 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:17,942 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,944 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:17,945 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:17,945 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:17,945 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983317945 2023-02-09 22:54:17,945 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983377945 2023-02-09 22:54:17,945 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 22:54:17,945 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:17,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN}] 2023-02-09 22:54:17,947 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:17,947 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:17,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:17,947 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:18,195 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,195 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:18,195 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:18,195 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:18,196 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:18,197 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:18,197 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:18,197 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:18,197 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:18,198 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:18,199 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 22:54:18,199 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:18,200 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:18,200 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:18,200 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:18,201 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,201 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:18,201 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,201 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,202 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,202 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:18,203 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,203 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,204 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:18,204 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,204 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:18,204 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,204 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,205 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,205 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:54:18,205 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:18,206 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:18,206 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983288206 2023-02-09 22:54:18,207 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:18,208 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,210 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:18,210 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:18,210 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:18,210 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983318210 2023-02-09 22:54:18,210 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983378210 2023-02-09 22:54:18,210 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:18,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN}] 2023-02-09 22:54:18,211 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:18,213 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,213 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:18,213 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983258213"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983258213"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983258213"}]},"ts":"1675983258213"} 2023-02-09 22:54:18,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:18,215 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:18,215 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:18,215 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:18,460 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,461 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:18,461 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:18,461 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:18,462 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:18,462 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:18,462 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:18,462 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:18,462 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:18,463 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:18,464 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:18,464 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 22:54:18,464 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:18,465 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:18,465 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:18,465 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:18,465 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,465 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:18,466 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,466 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,466 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,466 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:18,467 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,467 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,467 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:18,467 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,468 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,468 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:18,468 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,468 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,468 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,468 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:18,468 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,468 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:18,469 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:18,469 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:18,469 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:18,471 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983288471 2023-02-09 22:54:18,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,472 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:18,474 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:18,474 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSING, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:18,474 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:18,474 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983318474 2023-02-09 22:54:18,474 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983378474 2023-02-09 22:54:18,474 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:18,475 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:18,475 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983258475"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983258475"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983258475"}]},"ts":"1675983258475"} 2023-02-09 22:54:18,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:18,477 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:18,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:18,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing f0f398cb96c5ad5c92cff9427e7b68fd, disabling compactions & flushes 2023-02-09 22:54:18,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:18,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:18,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. after waiting 0 ms 2023-02-09 22:54:18,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:18,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing f0f398cb96c5ad5c92cff9427e7b68fd 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 22:54:18,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/4bb0ad7d76844b279b87b81e3d17a317 2023-02-09 22:54:18,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/8493574027c9469ca3409b055f94f943 2023-02-09 22:54:18,658 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/4bb0ad7d76844b279b87b81e3d17a317 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 2023-02-09 22:54:18,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:54:18,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/8493574027c9469ca3409b055f94f943 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 2023-02-09 22:54:18,666 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:54:18,667 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 37ms, sequenceid=82, compaction requested=true 2023-02-09 22:54:18,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 22:54:18,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:18,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:18,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for f0f398cb96c5ad5c92cff9427e7b68fd: Waiting for close lock at 1675983258630Running coprocessor pre-close hooks at 1675983258630Disabling compacts and flushes for region at 1675983258630Disabling writes for close at 1675983258630Obtaining lock to block concurrent updates at 1675983258631 (+1 ms)Preparing flush snapshotting stores in f0f398cb96c5ad5c92cff9427e7b68fd at 1675983258631Finished memstore snapshotting testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675983258631Flushing stores of testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. at 1675983258631Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf1: creating writer at 1675983258632 (+1 ms)Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf1: appending metadata at 1675983258634 (+2 ms)Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf1: closing flushed file at 1675983258634Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf2: creating writer at 1675983258645 (+11 ms)Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf2: appending metadata at 1675983258647 (+2 ms)Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf2: closing flushed file at 1675983258647Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf1: reopening flushed file at 1675983258658 (+11 ms)Flushing f0f398cb96c5ad5c92cff9427e7b68fd/cf2: reopening flushed file at 1675983258663 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f0f398cb96c5ad5c92cff9427e7b68fd in 37ms, sequenceid=82, compaction requested=true at 1675983258667 (+4 ms)Writing region close event to WAL at 1675983258669 (+2 ms)Running coprocessor post-close hooks at 1675983258672 (+3 ms)Closed at 1675983258672 2023-02-09 22:54:18,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:18,675 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED 2023-02-09 22:54:18,675 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983258674"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983258674"}]},"ts":"1675983258674"} 2023-02-09 22:54:18,676 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:18,676 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:18,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:18,676 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:18,725 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,725 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:18,725 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:18,725 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:18,727 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:18,727 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:18,727 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:18,727 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:18,727 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:18,728 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:18,729 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:18,729 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 22:54:18,729 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 22:54:18,729 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:18,729 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:18,729 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:18,730 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:18,730 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,730 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:18,730 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,730 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,731 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,731 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:18,731 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,731 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,732 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:18,732 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,732 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,732 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:18,732 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:18,733 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,733 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,733 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,733 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:18,733 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,733 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:18,733 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:18,734 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:18,735 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:18,735 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:18,735 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983288735 2023-02-09 22:54:18,736 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:18,736 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:18,738 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:18,739 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:18,739 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:18,739 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983318739 2023-02-09 22:54:18,739 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983378739 2023-02-09 22:54:18,739 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:18,739 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:18,740 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983258739"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983258739"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983258739"}]},"ts":"1675983258739"} 2023-02-09 22:54:18,741 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:18,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-09 22:54:18,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 in 264 msec 2023-02-09 22:54:18,744 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:18,745 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:18,745 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:18,989 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,990 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:18,990 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:18,990 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:18,990 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:18,990 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:18,990 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:18,990 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:18,990 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:18,991 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:18,992 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:18,992 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 22:54:18,992 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:18,993 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:18,993 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:18,993 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:18,993 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:18,993 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,994 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:18,994 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,994 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:18,994 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,995 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:18,995 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,995 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:18,995 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:18,996 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,996 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:18,996 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:18,996 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:18,996 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,996 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,997 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:18,997 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:18,997 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:18,997 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:18,997 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:18,998 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:18,998 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:18,998 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983288998 2023-02-09 22:54:18,999 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:18,999 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:19,001 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:19,001 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:19,001 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:19,001 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983319001 2023-02-09 22:54:19,001 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983379001 2023-02-09 22:54:19,001 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:19,002 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:19,003 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:19,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-09 22:54:19,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN in 793 msec 2023-02-09 22:54:19,008 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:19,008 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:19,008 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:19,252 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,252 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:19,252 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:19,252 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:19,252 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:19,252 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:19,252 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:19,253 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:19,253 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:19,254 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:19,255 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:19,255 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:19,255 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:19,255 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:19,256 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:19,256 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:19,256 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:19,257 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:19,257 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:19,257 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:19,257 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:19,258 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:19,258 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:19,258 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:19,259 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:19,259 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:19,260 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,260 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:19,260 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:19,260 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:19,260 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:19,260 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:19,261 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:19,264 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:19,269 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983289269 2023-02-09 22:54:19,270 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:19,271 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,273 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:19,273 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:19,274 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:19,274 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983319274 2023-02-09 22:54:19,274 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983379274 2023-02-09 22:54:19,274 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:19,274 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,274 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,274 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,279 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:19,288 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=f0f398cb96c5ad5c92cff9427e7b68fd, threads=14 2023-02-09 22:54:19,289 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,289 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,289 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,289 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,289 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,289 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,289 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,290 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,290 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,291 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,291 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,291 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,291 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,291 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,380 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,405 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,406 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,411 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,436 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,436 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,461 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,461 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,464 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,466 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,466 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,470 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,470 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,742 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:19,742 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region f0f398cb96c5ad5c92cff9427e7b68fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:54:19,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:19,783 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:19,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:19,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:19,905 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:20,024 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,025 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:20,025 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:20,025 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:20,026 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:20,026 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:20,026 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:20,026 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:20,026 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:20,027 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:20,028 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 22:54:20,028 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:20,028 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:20,028 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:20,028 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:20,029 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:20,029 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:20,029 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:20,029 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:20,029 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:20,030 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:20,030 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:20,030 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:20,030 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:20,030 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:20,031 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:20,031 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,031 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:20,031 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:20,031 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:20,031 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,031 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,032 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,032 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:20,032 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:20,032 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:20,032 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:20,033 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:20,033 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983290033 2023-02-09 22:54:20,033 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:20,035 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,037 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:20,037 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:20,037 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:20,037 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983320037 2023-02-09 22:54:20,037 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983380037 2023-02-09 22:54:20,037 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:54:20,045 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=f0f398cb96c5ad5c92cff9427e7b68fd, threads=14 2023-02-09 22:54:20,046 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,046 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,046 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,046 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,047 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,047 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,047 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,050 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,050 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,050 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,050 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,050 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,050 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,051 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,184 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,189 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,190 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,214 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,219 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,227 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,231 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,232 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,237 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,238 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,238 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,240 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,552 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,557 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e for region: f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:20,557 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region f0f398cb96c5ad5c92cff9427e7b68fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:54:20,590 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:20,595 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/9feae2fd1805a15b1414748d430e8cee/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:54:20,596 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/25e75757713f4d54468876cc23fd3fd9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:54:20,596 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:20,596 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:20,596 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:20,788 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,788 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:20,788 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:20,788 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:20,789 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:20,789 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:20,789 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:20,789 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:20,789 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:20,790 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:20,791 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 22:54:20,791 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:20,791 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:20,791 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:20,792 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:20,792 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:20,792 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:20,793 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:20,793 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:20,793 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:20,793 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:20,793 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:20,794 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:20,794 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:20,794 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:20,794 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:20,794 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,795 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:20,795 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:20,797 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:20,797 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,797 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,797 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,797 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:20,797 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:20,797 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 22:54:20,798 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:20,798 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:20,799 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983290799 2023-02-09 22:54:20,799 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:20,800 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,802 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:20,802 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:20,802 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:20,802 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983320802 2023-02-09 22:54:20,802 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983380802 2023-02-09 22:54:20,803 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:20,805 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:20,808 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:20,809 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:20,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:20,809 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:21,053 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,053 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:21,053 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:21,054 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:21,055 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:21,055 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:21,055 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:21,055 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:21,055 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:21,056 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:21,056 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 22:54:21,056 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:21,057 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:21,057 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:21,057 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:21,057 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:21,057 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:21,057 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,058 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:21,058 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,058 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,059 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,059 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:21,059 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,059 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,060 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:21,060 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,060 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:21,060 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:21,060 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:21,060 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,060 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,060 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,060 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,060 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,061 DEBUG [Listener at localhost/44743] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,061 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:21,061 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:21,062 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:21,062 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983291062 2023-02-09 22:54:21,062 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:21,065 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,066 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,067 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,067 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:21,067 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983321067 2023-02-09 22:54:21,067 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983381067 2023-02-09 22:54:21,067 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:54:21,067 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:21,081 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:21,081 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 22:54:21,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:21,081 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:21,318 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,318 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:21,318 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:21,318 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:21,319 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:21,319 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:21,319 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:21,319 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:21,319 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:21,320 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:21,321 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 22:54:21,321 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:21,321 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:21,322 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:21,322 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:21,322 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:21,322 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:21,323 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,323 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:21,323 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,324 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,324 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,324 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:21,324 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,324 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,324 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:21,325 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,325 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:21,325 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:21,325 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:21,325 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,326 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,326 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,326 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,326 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,326 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:21,326 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:21,327 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:21,327 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983291327 2023-02-09 22:54:21,327 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:21,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,333 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,333 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,333 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:21,333 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983321333 2023-02-09 22:54:21,333 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983381333 2023-02-09 22:54:21,334 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-09 22:54:21,334 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:21,336 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:21,336 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 22:54:21,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:21,336 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:21,585 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:21,585 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:21,586 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:21,587 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 22:54:21,587 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:21,587 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:21,587 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:21,588 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:21,588 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:21,588 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:21,588 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,588 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:21,588 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,588 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,589 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,589 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:21,589 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,589 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,589 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:21,590 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,590 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:21,590 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:21,590 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:21,590 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,591 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,591 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,591 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,591 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,591 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:21,591 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:21,592 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:21,592 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983291592 2023-02-09 22:54:21,593 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:21,594 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,596 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,596 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,596 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:21,596 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983321596 2023-02-09 22:54:21,596 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983381596 2023-02-09 22:54:21,596 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 22:54:21,596 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:21,598 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:21,598 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 22:54:21,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:21,599 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:21,846 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,847 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:21,847 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:21,847 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:21,848 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:21,848 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:21,848 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:21,848 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:21,848 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:21,849 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:21,849 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 22:54:21,849 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:21,849 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:21,850 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:21,850 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:21,850 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:21,850 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:21,851 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,851 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:21,851 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,851 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:21,851 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,851 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:21,852 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,852 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:21,852 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:21,852 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,852 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:21,852 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:21,853 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:21,853 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,853 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,853 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,853 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,853 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:21,853 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:21,854 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:21,854 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:21,854 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983291854 2023-02-09 22:54:21,854 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:21,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:21,858 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,858 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:21,858 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:21,858 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983321858 2023-02-09 22:54:21,858 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983381858 2023-02-09 22:54:21,858 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:21,858 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:21,862 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN] 2023-02-09 22:54:21,862 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:21,862 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 22:54:21,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:21,862 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:22,109 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,109 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:22,109 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:22,109 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:22,110 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:22,110 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:22,110 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:22,110 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:22,110 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:22,111 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:22,112 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 22:54:22,112 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:54:22,112 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:54:22,112 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 22:54:22,112 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:22,112 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:22,112 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:22,113 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:22,113 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,113 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:22,113 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,113 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,113 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,114 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:22,114 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,114 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,114 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:22,115 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,115 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:22,115 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:22,115 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,115 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:22,115 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,116 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,116 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,116 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:22,116 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:22,116 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:22,116 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:22,116 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:22,117 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:22,117 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:22,117 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983292117 2023-02-09 22:54:22,117 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:22,119 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,120 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,122 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,122 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,122 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:22,122 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983322122 2023-02-09 22:54:22,122 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983382122 2023-02-09 22:54:22,122 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:54:22,122 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:22,127 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:22,127 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 22:54:22,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:22,127 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:22,372 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,373 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:22,373 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:22,373 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:22,373 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:22,373 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:22,373 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:22,373 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:22,373 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:22,374 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:22,375 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:54:22,375 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 22:54:22,375 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:22,375 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:22,375 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:22,375 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:22,376 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,376 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:22,376 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,376 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,376 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,377 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:22,377 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,377 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,377 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:22,378 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,378 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,378 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:22,378 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,378 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,378 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,378 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:22,378 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:22,379 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:22,379 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:22,379 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:22,379 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983292379 2023-02-09 22:54:22,381 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:22,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,382 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,384 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,384 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,384 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:22,384 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983322384 2023-02-09 22:54:22,384 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983382384 2023-02-09 22:54:22,384 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:54:22,385 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:22,387 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:22,387 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 22:54:22,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:22,387 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:22,635 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,635 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:22,635 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:22,635 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:22,637 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:22,638 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:22,638 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:22,638 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:22,638 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:22,639 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:22,639 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 22:54:22,639 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 22:54:22,640 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:22,640 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:22,640 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:22,640 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:22,641 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,641 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:22,641 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,641 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,642 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,642 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:22,642 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,642 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,642 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:22,643 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,643 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,643 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:22,643 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,643 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,643 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,643 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:22,643 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 22:54:22,644 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:22,644 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:22,644 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:22,645 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983292645 2023-02-09 22:54:22,646 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:22,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,648 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,650 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,650 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,650 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:22,650 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983322650 2023-02-09 22:54:22,650 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983382650 2023-02-09 22:54:22,650 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:54:22,650 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:22,652 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:22,652 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 22:54:22,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:22,653 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:22,900 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,900 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:22,900 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:22,901 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:22,901 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:22,901 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:22,901 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:22,901 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:22,901 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:22,902 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:22,902 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 22:54:22,903 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 22:54:22,903 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:22,903 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:22,903 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:22,903 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:22,904 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,904 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:22,904 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,904 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:22,905 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,905 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:22,905 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,905 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:22,906 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:22,906 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,906 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,906 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:22,907 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,907 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,907 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,907 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:54:22,907 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 22:54:22,908 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:22,908 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:22,908 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:22,909 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983292909 2023-02-09 22:54:22,909 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:22,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:22,913 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,914 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,915 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:22,915 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:22,915 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983322915 2023-02-09 22:54:22,915 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983382915 2023-02-09 22:54:22,915 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-09 22:54:22,915 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:54:22,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 exec-time=5.4960 sec 2023-02-09 22:54:22,917 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:22,918 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN; state=CLOSED, location=jenkins-hbase9.apache.org,41885,1675983241428; forceNewPlan=false, retain=true 2023-02-09 22:54:23,068 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:23,069 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:23,070 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983263069"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983263069"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983263069"}]},"ts":"1675983263069"} 2023-02-09 22:54:23,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:23,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:54:23,072 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:23,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:54:23,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:54:23,165 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:23,165 WARN [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-09 22:54:23,165 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:54:23,165 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:54:23,165 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:54:23,166 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:54:23,166 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:54:23,166 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:54:23,166 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:54:23,166 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:54:23,167 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:54:23,167 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-09 22:54:23,168 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:54:23,168 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:54:23,168 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:54:23,168 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:54:23,168 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:23,168 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:54:23,169 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:23,169 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:54:23,169 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:23,169 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:54:23,169 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:23,169 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:54:23,169 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:23,170 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:54:23,170 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:23,170 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:54:23,170 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:23,170 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-09 22:54:23,171 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:54:23,171 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:54:23,171 INFO [Listener at localhost/44743] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd to restore RIT 2023-02-09 22:54:23,171 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983293171 2023-02-09 22:54:23,172 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:54:23,172 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:54:23,174 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:23,174 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPENING, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:54:23,175 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:54:23,175 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983323175 2023-02-09 22:54:23,175 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983383175 2023-02-09 22:54:23,175 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:54:23,175 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:23,175 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983263175"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983263175"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983263175"}]},"ts":"1675983263175"} 2023-02-09 22:54:23,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:23,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f0f398cb96c5ad5c92cff9427e7b68fd, NAME => 'testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:23,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:23,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:23,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,334 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,335 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1 2023-02-09 22:54:23,335 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1 2023-02-09 22:54:23,336 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0f398cb96c5ad5c92cff9427e7b68fd columnFamilyName cf1 2023-02-09 22:54:23,341 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f 2023-02-09 22:54:23,345 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 2023-02-09 22:54:23,348 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 2023-02-09 22:54:23,351 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 2023-02-09 22:54:23,354 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 2023-02-09 22:54:23,358 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c 2023-02-09 22:54:23,361 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 2023-02-09 22:54:23,361 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(310): Store=f0f398cb96c5ad5c92cff9427e7b68fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:23,361 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,362 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2 2023-02-09 22:54:23,362 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2 2023-02-09 22:54:23,362 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0f398cb96c5ad5c92cff9427e7b68fd columnFamilyName cf2 2023-02-09 22:54:23,368 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 2023-02-09 22:54:23,371 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf 2023-02-09 22:54:23,374 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 2023-02-09 22:54:23,377 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e 2023-02-09 22:54:23,380 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 2023-02-09 22:54:23,383 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 2023-02-09 22:54:23,386 DEBUG [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d 2023-02-09 22:54:23,386 INFO [StoreOpener-f0f398cb96c5ad5c92cff9427e7b68fd-1] regionserver.HStore(310): Store=f0f398cb96c5ad5c92cff9427e7b68fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:23,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.splits 2023-02-09 22:54:23,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:23,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened f0f398cb96c5ad5c92cff9427e7b68fd; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62217098, jitterRate=-0.07289299368858337}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:23,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for f0f398cb96c5ad5c92cff9427e7b68fd: Running coprocessor pre-open hook at 1675983263333Writing region info on filesystem at 1675983263333Initializing all the Stores at 1675983263334 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983263334Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983263334Cleaning up temporary data from old regions at 1675983263388 (+54 ms)Cleaning up detritus from prior splits at 1675983263388Running coprocessor post-open hooks at 1675983263391 (+3 ms)Region opened successfully at 1675983263391 2023-02-09 22:54:23,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd., pid=48, masterSystemTime=1675983263329 2023-02-09 22:54:23,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:23,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:23,393 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:23,393 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:23,394 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 22:54:23,394 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 22:54:23,394 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.HStore(1912): f0f398cb96c5ad5c92cff9427e7b68fd/cf1 is initiating minor compaction (all files) 2023-02-09 22:54:23,395 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.HStore(1912): f0f398cb96c5ad5c92cff9427e7b68fd/cf2 is initiating minor compaction (all files) 2023-02-09 22:54:23,395 INFO [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f0f398cb96c5ad5c92cff9427e7b68fd/cf1 in testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,395 INFO [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f0f398cb96c5ad5c92cff9427e7b68fd/cf2 in testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,395 INFO [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp, totalSize=34.9 K 2023-02-09 22:54:23,396 INFO [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp, totalSize=34.9 K 2023-02-09 22:54:23,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,396 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.Compactor(207): Compacting 5443cf4e699145198d0cf0313fec8a79, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983255946 2023-02-09 22:54:23,396 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.Compactor(207): Compacting 96b3675d98f942068ef1101e13fef51e, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983255946 2023-02-09 22:54:23,396 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:23,396 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983263396"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983263396"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983263396"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983263396"}]},"ts":"1675983263396"} 2023-02-09 22:54:23,396 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.Compactor(207): Compacting 97e3ebd578dd48f69ef0a16fddf97617, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983256133 2023-02-09 22:54:23,396 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.Compactor(207): Compacting e49f24bfca344b5b856595fa4f952f5d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983256133 2023-02-09 22:54:23,397 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.Compactor(207): Compacting 5111ff6c32bd449482e6e2900efdfff5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983256412 2023-02-09 22:54:23,397 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.Compactor(207): Compacting 90dd446abaed402ba45bf28f11399baf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983256412 2023-02-09 22:54:23,397 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.Compactor(207): Compacting 1b4348cf3799482c897a1e8fd618c04f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983256601 2023-02-09 22:54:23,397 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.Compactor(207): Compacting 938e4a6cfed349deba92dfbeef1c6ac3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983256601 2023-02-09 22:54:23,398 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.Compactor(207): Compacting 66bf557d13234f70a611073d685cef29, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983256781 2023-02-09 22:54:23,398 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.Compactor(207): Compacting dec37788d2ff4ce9a9247787f2e8a650, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983256781 2023-02-09 22:54:23,398 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.Compactor(207): Compacting 8b7790ab4c894920af68a72e08b6fa8c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983256963 2023-02-09 22:54:23,398 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.Compactor(207): Compacting acf1c408ee5b4cf5bd529bebb06bd683, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983256963 2023-02-09 22:54:23,398 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] compactions.Compactor(207): Compacting 4bb0ad7d76844b279b87b81e3d17a317, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983257158 2023-02-09 22:54:23,399 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] compactions.Compactor(207): Compacting 8493574027c9469ca3409b055f94f943, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983257158 2023-02-09 22:54:23,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-09 22:54:23,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 in 222 msec 2023-02-09 22:54:23,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN in 1.5390 sec 2023-02-09 22:54:23,428 INFO [RS:0;jenkins-hbase9:41885-longCompactions-0] throttle.PressureAwareThroughputController(145): f0f398cb96c5ad5c92cff9427e7b68fd#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:23,428 INFO [RS:0;jenkins-hbase9:41885-shortCompactions-0] throttle.PressureAwareThroughputController(145): f0f398cb96c5ad5c92cff9427e7b68fd#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:23,447 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf2/c8d2e7fc59ff4d9ca299ff8b9208c950 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/c8d2e7fc59ff4d9ca299ff8b9208c950 2023-02-09 22:54:23,448 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/.tmp/cf1/5a7e95b68dc440f680a387c123cca718 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5a7e95b68dc440f680a387c123cca718 2023-02-09 22:54:23,453 INFO [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f0f398cb96c5ad5c92cff9427e7b68fd/cf1 of f0f398cb96c5ad5c92cff9427e7b68fd into 5a7e95b68dc440f680a387c123cca718(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:23,453 INFO [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f0f398cb96c5ad5c92cff9427e7b68fd/cf2 of f0f398cb96c5ad5c92cff9427e7b68fd into c8d2e7fc59ff4d9ca299ff8b9208c950(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:23,453 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:23,453 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f0f398cb96c5ad5c92cff9427e7b68fd: 2023-02-09 22:54:23,453 INFO [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd., storeName=f0f398cb96c5ad5c92cff9427e7b68fd/cf1, priority=9, startTime=1675983263392; duration=0sec 2023-02-09 22:54:23,453 INFO [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd., storeName=f0f398cb96c5ad5c92cff9427e7b68fd/cf2, priority=9, startTime=1675983263393; duration=0sec 2023-02-09 22:54:23,455 DEBUG [RS:0;jenkins-hbase9:41885-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:23,455 DEBUG [RS:0;jenkins-hbase9:41885-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:23,752 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-09 22:54:23,754 DEBUG [Listener at localhost/44743] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-09 22:54:23,755 INFO [Listener at localhost/44743] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 22:54:23,757 INFO [Listener at localhost/44743] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-09 22:54:23,758 INFO [Listener at localhost/44743] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 22:54:23,758 INFO [Listener at localhost/44743] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-09 22:54:23,778 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:23,779 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-09 22:54:23,780 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testRollbackAndDoubleExecution 2023-02-09 22:54:23,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:23,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 22:54:23,784 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983263783"}]},"ts":"1675983263783"} 2023-02-09 22:54:23,785 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 22:54:23,786 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-09 22:54:23,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN}] 2023-02-09 22:54:23,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN 2023-02-09 22:54:23,788 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:23,788 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983263788"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983263788"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983263788"}]},"ts":"1675983263788"} 2023-02-09 22:54:23,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:23,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 22:54:23,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing f0f398cb96c5ad5c92cff9427e7b68fd, disabling compactions & flushes 2023-02-09 22:54:23,942 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. after waiting 0 ms 2023-02-09 22:54:23,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,942 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317] to archive 2023-02-09 22:54:23,943 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:23,945 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5443cf4e699145198d0cf0313fec8a79 2023-02-09 22:54:23,946 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/97e3ebd578dd48f69ef0a16fddf97617 2023-02-09 22:54:23,947 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5111ff6c32bd449482e6e2900efdfff5 2023-02-09 22:54:23,948 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/1b4348cf3799482c897a1e8fd618c04f 2023-02-09 22:54:23,949 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/66bf557d13234f70a611073d685cef29 2023-02-09 22:54:23,951 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/8b7790ab4c894920af68a72e08b6fa8c 2023-02-09 22:54:23,952 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/4bb0ad7d76844b279b87b81e3d17a317 2023-02-09 22:54:23,957 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943] to archive 2023-02-09 22:54:23,957 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:23,959 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/96b3675d98f942068ef1101e13fef51e 2023-02-09 22:54:23,960 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/e49f24bfca344b5b856595fa4f952f5d 2023-02-09 22:54:23,961 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/90dd446abaed402ba45bf28f11399baf 2023-02-09 22:54:23,962 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/938e4a6cfed349deba92dfbeef1c6ac3 2023-02-09 22:54:23,963 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/dec37788d2ff4ce9a9247787f2e8a650 2023-02-09 22:54:23,964 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/acf1c408ee5b4cf5bd529bebb06bd683 2023-02-09 22:54:23,964 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/8493574027c9469ca3409b055f94f943 2023-02-09 22:54:23,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:54:23,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:23,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd. 2023-02-09 22:54:23,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for f0f398cb96c5ad5c92cff9427e7b68fd: Waiting for close lock at 1675983263941Running coprocessor pre-close hooks at 1675983263941Disabling compacts and flushes for region at 1675983263941Disabling writes for close at 1675983263942 (+1 ms)Writing region close event to WAL at 1675983263965 (+23 ms)Running coprocessor post-close hooks at 1675983263969 (+4 ms)Closed at 1675983263969 2023-02-09 22:54:23,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:23,971 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=f0f398cb96c5ad5c92cff9427e7b68fd, regionState=CLOSED 2023-02-09 22:54:23,971 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983263971"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983263971"}]},"ts":"1675983263971"} 2023-02-09 22:54:23,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-09 22:54:23,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure f0f398cb96c5ad5c92cff9427e7b68fd, server=jenkins-hbase9.apache.org,41885,1675983241428 in 182 msec 2023-02-09 22:54:23,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-09 22:54:23,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, UNASSIGN in 187 msec 2023-02-09 22:54:23,976 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983263976"}]},"ts":"1675983263976"} 2023-02-09 22:54:23,977 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 22:54:23,978 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-09 22:54:23,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 199 msec 2023-02-09 22:54:24,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 22:54:24,086 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-09 22:54:24,087 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testRollbackAndDoubleExecution 2023-02-09 22:54:24,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:24,089 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:24,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 22:54:24,090 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:24,093 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:24,095 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/recovered.edits] 2023-02-09 22:54:24,098 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5a7e95b68dc440f680a387c123cca718 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf1/5a7e95b68dc440f680a387c123cca718 2023-02-09 22:54:24,100 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/c8d2e7fc59ff4d9ca299ff8b9208c950 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/cf2/c8d2e7fc59ff4d9ca299ff8b9208c950 2023-02-09 22:54:24,102 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd/recovered.edits/90.seqid 2023-02-09 22:54:24,103 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackAndDoubleExecution/f0f398cb96c5ad5c92cff9427e7b68fd 2023-02-09 22:54:24,103 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 22:54:24,106 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:24,108 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-09 22:54:24,109 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-09 22:54:24,110 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:24,110 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-09 22:54:24,110 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983264110"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:24,111 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 22:54:24,112 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f0f398cb96c5ad5c92cff9427e7b68fd, NAME => 'testRollbackAndDoubleExecution,,1675983255438.f0f398cb96c5ad5c92cff9427e7b68fd.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 22:54:24,112 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-09 22:54:24,112 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983264112"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:24,113 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-09 22:54:24,115 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:54:24,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 28 msec 2023-02-09 22:54:24,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 22:54:24,191 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-09 22:54:24,206 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=404 (was 403) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:46986 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:47020 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50524 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:46972 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50546 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase9:41885-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50476 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:46976 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50578 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50554 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:46996 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50490 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50500 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50474 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50538 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:47008 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1588583819_15 at /127.0.0.1:50510 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=867 (was 856) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 222) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=10931 (was 11032) 2023-02-09 22:54:24,220 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=404, OpenFileDescriptor=867, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=168, AvailableMemoryMB=10930 2023-02-09 22:54:24,220 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:24,224 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:54:24,224 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:54:24,226 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:24,227 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:24,228 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:24,229 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,230 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213 empty. 2023-02-09 22:54:24,230 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,230 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 22:54:24,244 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:24,245 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 57fde352bce7caf7c73c967d7b39b213, NAME => 'testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:54:24,262 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:24,262 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 57fde352bce7caf7c73c967d7b39b213, disabling compactions & flushes 2023-02-09 22:54:24,262 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,262 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,262 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. after waiting 0 ms 2023-02-09 22:54:24,262 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,262 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,262 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 57fde352bce7caf7c73c967d7b39b213: Waiting for close lock at 1675983264262Disabling compacts and flushes for region at 1675983264262Disabling writes for close at 1675983264262Writing region close event to WAL at 1675983264262Closed at 1675983264262 2023-02-09 22:54:24,264 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:24,265 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983264265"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983264265"}]},"ts":"1675983264265"} 2023-02-09 22:54:24,267 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:24,268 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:24,268 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983264268"}]},"ts":"1675983264268"} 2023-02-09 22:54:24,269 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-09 22:54:24,272 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:24,273 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:24,273 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:24,273 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:24,273 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:24,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57fde352bce7caf7c73c967d7b39b213, ASSIGN}] 2023-02-09 22:54:24,275 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57fde352bce7caf7c73c967d7b39b213, ASSIGN 2023-02-09 22:54:24,275 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57fde352bce7caf7c73c967d7b39b213, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41885,1675983241428; forceNewPlan=false, retain=false 2023-02-09 22:54:24,425 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:24,427 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=57fde352bce7caf7c73c967d7b39b213, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:24,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983264427"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983264427"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983264427"}]},"ts":"1675983264427"} 2023-02-09 22:54:24,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 57fde352bce7caf7c73c967d7b39b213, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:24,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 57fde352bce7caf7c73c967d7b39b213, NAME => 'testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:24,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:24,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:24,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,585 INFO [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,586 DEBUG [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/cf1 2023-02-09 22:54:24,586 DEBUG [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/cf1 2023-02-09 22:54:24,587 INFO [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57fde352bce7caf7c73c967d7b39b213 columnFamilyName cf1 2023-02-09 22:54:24,587 INFO [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] regionserver.HStore(310): Store=57fde352bce7caf7c73c967d7b39b213/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:24,587 INFO [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,589 DEBUG [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/cf2 2023-02-09 22:54:24,589 DEBUG [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/cf2 2023-02-09 22:54:24,589 INFO [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57fde352bce7caf7c73c967d7b39b213 columnFamilyName cf2 2023-02-09 22:54:24,589 INFO [StoreOpener-57fde352bce7caf7c73c967d7b39b213-1] regionserver.HStore(310): Store=57fde352bce7caf7c73c967d7b39b213/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:24,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:24,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:24,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 57fde352bce7caf7c73c967d7b39b213; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63909354, jitterRate=-0.047676414251327515}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:24,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 57fde352bce7caf7c73c967d7b39b213: Running coprocessor pre-open hook at 1675983264584Writing region info on filesystem at 1675983264584Initializing all the Stores at 1675983264585 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983264585Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983264585Cleaning up temporary data from old regions at 1675983264591 (+6 ms)Cleaning up detritus from prior splits at 1675983264592 (+1 ms)Running coprocessor post-open hooks at 1675983264596 (+4 ms)Region opened successfully at 1675983264596 2023-02-09 22:54:24,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213., pid=55, masterSystemTime=1675983264580 2023-02-09 22:54:24,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,599 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=57fde352bce7caf7c73c967d7b39b213, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:24,599 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983264599"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983264599"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983264599"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983264599"}]},"ts":"1675983264599"} 2023-02-09 22:54:24,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-09 22:54:24,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 57fde352bce7caf7c73c967d7b39b213, server=jenkins-hbase9.apache.org,41885,1675983241428 in 173 msec 2023-02-09 22:54:24,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-09 22:54:24,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57fde352bce7caf7c73c967d7b39b213, ASSIGN in 329 msec 2023-02-09 22:54:24,605 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:24,605 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983264605"}]},"ts":"1675983264605"} 2023-02-09 22:54:24,606 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-09 22:54:24,608 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:24,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 383 msec 2023-02-09 22:54:24,727 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:24,728 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:24,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:24,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:24,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:24,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57fde352bce7caf7c73c967d7b39b213, UNASSIGN}] 2023-02-09 22:54:24,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57fde352bce7caf7c73c967d7b39b213, UNASSIGN 2023-02-09 22:54:24,732 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=57fde352bce7caf7c73c967d7b39b213, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:24,732 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983264732"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983264732"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983264732"}]},"ts":"1675983264732"} 2023-02-09 22:54:24,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 57fde352bce7caf7c73c967d7b39b213, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:24,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 57fde352bce7caf7c73c967d7b39b213, disabling compactions & flushes 2023-02-09 22:54:24,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. after waiting 0 ms 2023-02-09 22:54:24,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 22:54:24,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:24,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213. 2023-02-09 22:54:24,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 57fde352bce7caf7c73c967d7b39b213: Waiting for close lock at 1675983264886Running coprocessor pre-close hooks at 1675983264886Disabling compacts and flushes for region at 1675983264886Disabling writes for close at 1675983264887 (+1 ms)Writing region close event to WAL at 1675983264887Running coprocessor post-close hooks at 1675983264891 (+4 ms)Closed at 1675983264891 2023-02-09 22:54:24,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:24,893 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=57fde352bce7caf7c73c967d7b39b213, regionState=CLOSED 2023-02-09 22:54:24,893 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983264893"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983264893"}]},"ts":"1675983264893"} 2023-02-09 22:54:24,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-09 22:54:24,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 57fde352bce7caf7c73c967d7b39b213, server=jenkins-hbase9.apache.org,41885,1675983241428 in 160 msec 2023-02-09 22:54:24,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-09 22:54:24,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57fde352bce7caf7c73c967d7b39b213, UNASSIGN in 165 msec 2023-02-09 22:54:24,920 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 22:54:24,922 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 22:54:24,923 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983264923"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675983264923"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675983264923"}]},"ts":"1675983264923"} 2023-02-09 22:54:24,923 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983264923"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983264923"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983264923"}]},"ts":"1675983264923"} 2023-02-09 22:54:24,923 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983264923"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983264923"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983264923"}]},"ts":"1675983264923"} 2023-02-09 22:54:24,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8159207c39763434e943cce3d99a0ffb, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dbacfa106d4f2da40649a61989eade8a, ASSIGN}] 2023-02-09 22:54:24,927 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dbacfa106d4f2da40649a61989eade8a, ASSIGN 2023-02-09 22:54:24,928 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dbacfa106d4f2da40649a61989eade8a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,41885,1675983241428; forceNewPlan=false, retain=false 2023-02-09 22:54:24,928 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8159207c39763434e943cce3d99a0ffb, ASSIGN 2023-02-09 22:54:24,929 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8159207c39763434e943cce3d99a0ffb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,41885,1675983241428; forceNewPlan=false, retain=false 2023-02-09 22:54:25,078 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 22:54:25,080 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=8159207c39763434e943cce3d99a0ffb, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:25,080 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265080"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983265080"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983265080"}]},"ts":"1675983265080"} 2023-02-09 22:54:25,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 8159207c39763434e943cce3d99a0ffb, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:25,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=dbacfa106d4f2da40649a61989eade8a, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:25,084 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265084"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983265084"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983265084"}]},"ts":"1675983265084"} 2023-02-09 22:54:25,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure dbacfa106d4f2da40649a61989eade8a, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:25,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. 2023-02-09 22:54:25,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dbacfa106d4f2da40649a61989eade8a, NAME => 'testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 22:54:25,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:25,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:25,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,238 INFO [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,240 DEBUG [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/cf1 2023-02-09 22:54:25,240 DEBUG [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/cf1 2023-02-09 22:54:25,240 INFO [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dbacfa106d4f2da40649a61989eade8a columnFamilyName cf1 2023-02-09 22:54:25,241 INFO [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] regionserver.HStore(310): Store=dbacfa106d4f2da40649a61989eade8a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:25,241 INFO [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,242 DEBUG [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/cf2 2023-02-09 22:54:25,242 DEBUG [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/cf2 2023-02-09 22:54:25,242 INFO [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dbacfa106d4f2da40649a61989eade8a columnFamilyName cf2 2023-02-09 22:54:25,243 INFO [StoreOpener-dbacfa106d4f2da40649a61989eade8a-1] regionserver.HStore(310): Store=dbacfa106d4f2da40649a61989eade8a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:25,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:25,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened dbacfa106d4f2da40649a61989eade8a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60973742, jitterRate=-0.09142044186592102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:25,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for dbacfa106d4f2da40649a61989eade8a: Running coprocessor pre-open hook at 1675983265237Writing region info on filesystem at 1675983265237Initializing all the Stores at 1675983265238 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983265238Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983265238Cleaning up temporary data from old regions at 1675983265245 (+7 ms)Cleaning up detritus from prior splits at 1675983265246 (+1 ms)Running coprocessor post-open hooks at 1675983265249 (+3 ms)Region opened successfully at 1675983265249 2023-02-09 22:54:25,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a., pid=62, masterSystemTime=1675983265233 2023-02-09 22:54:25,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. 2023-02-09 22:54:25,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. 2023-02-09 22:54:25,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. 2023-02-09 22:54:25,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8159207c39763434e943cce3d99a0ffb, NAME => 'testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 22:54:25,252 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=dbacfa106d4f2da40649a61989eade8a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:25,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:25,252 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265252"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983265252"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983265252"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983265252"}]},"ts":"1675983265252"} 2023-02-09 22:54:25,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:25,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,253 INFO [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-09 22:54:25,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure dbacfa106d4f2da40649a61989eade8a, server=jenkins-hbase9.apache.org,41885,1675983241428 in 168 msec 2023-02-09 22:54:25,255 DEBUG [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/cf1 2023-02-09 22:54:25,255 DEBUG [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/cf1 2023-02-09 22:54:25,255 INFO [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8159207c39763434e943cce3d99a0ffb columnFamilyName cf1 2023-02-09 22:54:25,256 INFO [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] regionserver.HStore(310): Store=8159207c39763434e943cce3d99a0ffb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:25,256 INFO [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dbacfa106d4f2da40649a61989eade8a, ASSIGN in 329 msec 2023-02-09 22:54:25,257 DEBUG [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/cf2 2023-02-09 22:54:25,257 DEBUG [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/cf2 2023-02-09 22:54:25,258 INFO [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8159207c39763434e943cce3d99a0ffb columnFamilyName cf2 2023-02-09 22:54:25,258 INFO [StoreOpener-8159207c39763434e943cce3d99a0ffb-1] regionserver.HStore(310): Store=8159207c39763434e943cce3d99a0ffb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:25,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:25,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 8159207c39763434e943cce3d99a0ffb; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72380009, jitterRate=0.0785461813211441}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:25,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 8159207c39763434e943cce3d99a0ffb: Running coprocessor pre-open hook at 1675983265252Writing region info on filesystem at 1675983265252Initializing all the Stores at 1675983265253 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983265253Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983265253Cleaning up temporary data from old regions at 1675983265260 (+7 ms)Cleaning up detritus from prior splits at 1675983265261 (+1 ms)Running coprocessor post-open hooks at 1675983265263 (+2 ms)Region opened successfully at 1675983265263 2023-02-09 22:54:25,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb., pid=61, masterSystemTime=1675983265233 2023-02-09 22:54:25,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. 2023-02-09 22:54:25,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. 2023-02-09 22:54:25,266 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=8159207c39763434e943cce3d99a0ffb, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:25,266 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265266"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983265266"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983265266"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983265266"}]},"ts":"1675983265266"} 2023-02-09 22:54:25,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-09 22:54:25,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 8159207c39763434e943cce3d99a0ffb, server=jenkins-hbase9.apache.org,41885,1675983241428 in 185 msec 2023-02-09 22:54:25,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-09 22:54:25,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8159207c39763434e943cce3d99a0ffb, ASSIGN in 343 msec 2023-02-09 22:54:25,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a in 543 msec 2023-02-09 22:54:25,482 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 22:54:25,485 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:25,486 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,487 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 22:54:25,490 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983265490"}]},"ts":"1675983265490"} 2023-02-09 22:54:25,491 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-09 22:54:25,493 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-09 22:54:25,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8159207c39763434e943cce3d99a0ffb, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dbacfa106d4f2da40649a61989eade8a, UNASSIGN}] 2023-02-09 22:54:25,494 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dbacfa106d4f2da40649a61989eade8a, UNASSIGN 2023-02-09 22:54:25,495 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=dbacfa106d4f2da40649a61989eade8a, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:25,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265494"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983265494"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983265494"}]},"ts":"1675983265494"} 2023-02-09 22:54:25,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure dbacfa106d4f2da40649a61989eade8a, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:25,498 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8159207c39763434e943cce3d99a0ffb, UNASSIGN 2023-02-09 22:54:25,498 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=8159207c39763434e943cce3d99a0ffb, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:25,498 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265498"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983265498"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983265498"}]},"ts":"1675983265498"} 2023-02-09 22:54:25,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 8159207c39763434e943cce3d99a0ffb, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:25,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 22:54:25,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 8159207c39763434e943cce3d99a0ffb, disabling compactions & flushes 2023-02-09 22:54:25,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. 2023-02-09 22:54:25,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. 2023-02-09 22:54:25,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. after waiting 0 ms 2023-02-09 22:54:25,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. 2023-02-09 22:54:25,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 22:54:25,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:25,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb. 2023-02-09 22:54:25,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 8159207c39763434e943cce3d99a0ffb: Waiting for close lock at 1675983265648Running coprocessor pre-close hooks at 1675983265648Disabling compacts and flushes for region at 1675983265648Disabling writes for close at 1675983265649 (+1 ms)Writing region close event to WAL at 1675983265649Running coprocessor post-close hooks at 1675983265653 (+4 ms)Closed at 1675983265653 2023-02-09 22:54:25,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing dbacfa106d4f2da40649a61989eade8a, disabling compactions & flushes 2023-02-09 22:54:25,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. 2023-02-09 22:54:25,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. 2023-02-09 22:54:25,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. after waiting 0 ms 2023-02-09 22:54:25,655 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=8159207c39763434e943cce3d99a0ffb, regionState=CLOSED 2023-02-09 22:54:25,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. 2023-02-09 22:54:25,656 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265655"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983265655"}]},"ts":"1675983265655"} 2023-02-09 22:54:25,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-09 22:54:25,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 8159207c39763434e943cce3d99a0ffb, server=jenkins-hbase9.apache.org,41885,1675983241428 in 157 msec 2023-02-09 22:54:25,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 22:54:25,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:25,659 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a. 2023-02-09 22:54:25,659 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for dbacfa106d4f2da40649a61989eade8a: Waiting for close lock at 1675983265655Running coprocessor pre-close hooks at 1675983265655Disabling compacts and flushes for region at 1675983265655Disabling writes for close at 1675983265656 (+1 ms)Writing region close event to WAL at 1675983265656Running coprocessor post-close hooks at 1675983265659 (+3 ms)Closed at 1675983265659 2023-02-09 22:54:25,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8159207c39763434e943cce3d99a0ffb, UNASSIGN in 166 msec 2023-02-09 22:54:25,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,661 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=dbacfa106d4f2da40649a61989eade8a, regionState=CLOSED 2023-02-09 22:54:25,661 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983265661"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983265661"}]},"ts":"1675983265661"} 2023-02-09 22:54:25,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-09 22:54:25,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure dbacfa106d4f2da40649a61989eade8a, server=jenkins-hbase9.apache.org,41885,1675983241428 in 166 msec 2023-02-09 22:54:25,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-09 22:54:25,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=dbacfa106d4f2da40649a61989eade8a, UNASSIGN in 170 msec 2023-02-09 22:54:25,665 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983265665"}]},"ts":"1675983265665"} 2023-02-09 22:54:25,666 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-09 22:54:25,668 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-09 22:54:25,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 181 msec 2023-02-09 22:54:25,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 22:54:25,792 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-09 22:54:25,793 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,795 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 22:54:25,796 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,801 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:25,801 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,801 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,802 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/recovered.edits] 2023-02-09 22:54:25,803 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/recovered.edits] 2023-02-09 22:54:25,803 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/recovered.edits] 2023-02-09 22:54:25,808 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/recovered.edits/7.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a/recovered.edits/7.seqid 2023-02-09 22:54:25,809 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/recovered.edits/7.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb/recovered.edits/7.seqid 2023-02-09 22:54:25,809 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/recovered.edits/4.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213/recovered.edits/4.seqid 2023-02-09 22:54:25,809 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:54:25,809 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/8159207c39763434e943cce3d99a0ffb 2023-02-09 22:54:25,809 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionNoStoreFile/57fde352bce7caf7c73c967d7b39b213 2023-02-09 22:54:25,809 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 22:54:25,812 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,813 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-09 22:54:25,815 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-09 22:54:25,816 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,816 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-09 22:54:25,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983265816"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:25,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983265816"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:25,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983265816"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:25,818 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:54:25,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 57fde352bce7caf7c73c967d7b39b213, NAME => 'testSplitTableRegionNoStoreFile,,1675983264225.57fde352bce7caf7c73c967d7b39b213.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 8159207c39763434e943cce3d99a0ffb, NAME => 'testSplitTableRegionNoStoreFile,,1675983264727.8159207c39763434e943cce3d99a0ffb.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => dbacfa106d4f2da40649a61989eade8a, NAME => 'testSplitTableRegionNoStoreFile,41,1675983264727.dbacfa106d4f2da40649a61989eade8a.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 22:54:25,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-09 22:54:25,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983265818"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:25,819 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-09 22:54:25,821 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:54:25,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 28 msec 2023-02-09 22:54:25,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 22:54:25,897 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-09 22:54:25,911 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=407 (was 404) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1412667219_15 at /127.0.0.1:46986 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1412667219_15 at /127.0.0.1:46996 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=865 (was 867), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 229), ProcessCount=168 (was 168), AvailableMemoryMB=10927 (was 10930) 2023-02-09 22:54:25,925 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=407, OpenFileDescriptor=865, MaxFileDescriptor=60000, SystemLoadAverage=210, ProcessCount=168, AvailableMemoryMB=10926 2023-02-09 22:54:25,925 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:25,928 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:54:25,929 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:54:25,930 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:25,931 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:25,932 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:25,934 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:25,934 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2 empty. 2023-02-09 22:54:25,935 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:25,935 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 22:54:25,946 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:25,947 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => e757cfcd8ee3a29c09867ad25a0805c2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:54:25,956 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:25,956 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing e757cfcd8ee3a29c09867ad25a0805c2, disabling compactions & flushes 2023-02-09 22:54:25,956 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:25,956 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:25,956 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. after waiting 0 ms 2023-02-09 22:54:25,956 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:25,956 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:25,956 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for e757cfcd8ee3a29c09867ad25a0805c2: Waiting for close lock at 1675983265956Disabling compacts and flushes for region at 1675983265956Disabling writes for close at 1675983265956Writing region close event to WAL at 1675983265956Closed at 1675983265956 2023-02-09 22:54:25,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:25,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675983265959"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983265959"}]},"ts":"1675983265959"} 2023-02-09 22:54:25,960 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:25,961 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:25,961 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983265961"}]},"ts":"1675983265961"} 2023-02-09 22:54:25,962 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-09 22:54:25,965 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:25,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:25,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:25,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:25,967 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:25,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e757cfcd8ee3a29c09867ad25a0805c2, ASSIGN}] 2023-02-09 22:54:25,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e757cfcd8ee3a29c09867ad25a0805c2, ASSIGN 2023-02-09 22:54:25,969 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e757cfcd8ee3a29c09867ad25a0805c2, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:54:26,119 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:26,120 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e757cfcd8ee3a29c09867ad25a0805c2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675983266120"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983266120"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983266120"}]},"ts":"1675983266120"} 2023-02-09 22:54:26,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure e757cfcd8ee3a29c09867ad25a0805c2, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:26,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e757cfcd8ee3a29c09867ad25a0805c2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:26,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:26,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:26,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,282 INFO [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,283 DEBUG [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1 2023-02-09 22:54:26,284 DEBUG [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1 2023-02-09 22:54:26,284 INFO [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e757cfcd8ee3a29c09867ad25a0805c2 columnFamilyName cf1 2023-02-09 22:54:26,285 INFO [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] regionserver.HStore(310): Store=e757cfcd8ee3a29c09867ad25a0805c2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:26,285 INFO [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,287 DEBUG [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2 2023-02-09 22:54:26,287 DEBUG [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2 2023-02-09 22:54:26,287 INFO [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e757cfcd8ee3a29c09867ad25a0805c2 columnFamilyName cf2 2023-02-09 22:54:26,288 INFO [StoreOpener-e757cfcd8ee3a29c09867ad25a0805c2-1] regionserver.HStore(310): Store=e757cfcd8ee3a29c09867ad25a0805c2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:26,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:26,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:26,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:26,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened e757cfcd8ee3a29c09867ad25a0805c2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63214102, jitterRate=-0.058036476373672485}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:26,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for e757cfcd8ee3a29c09867ad25a0805c2: Running coprocessor pre-open hook at 1675983266281Writing region info on filesystem at 1675983266281Initializing all the Stores at 1675983266282 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983266282Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983266282Cleaning up temporary data from old regions at 1675983266290 (+8 ms)Cleaning up detritus from prior splits at 1675983266290Running coprocessor post-open hooks at 1675983266296 (+6 ms)Region opened successfully at 1675983266296 2023-02-09 22:54:26,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2., pid=71, masterSystemTime=1675983266274 2023-02-09 22:54:26,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,299 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e757cfcd8ee3a29c09867ad25a0805c2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675983266299"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983266299"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983266299"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983266299"}]},"ts":"1675983266299"} 2023-02-09 22:54:26,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-09 22:54:26,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure e757cfcd8ee3a29c09867ad25a0805c2, server=jenkins-hbase9.apache.org,40413,1675983241505 in 179 msec 2023-02-09 22:54:26,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-09 22:54:26,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e757cfcd8ee3a29c09867ad25a0805c2, ASSIGN in 336 msec 2023-02-09 22:54:26,305 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:26,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983266305"}]},"ts":"1675983266305"} 2023-02-09 22:54:26,306 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-09 22:54:26,308 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:26,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 379 msec 2023-02-09 22:54:26,438 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:26,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,440 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:26,440 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,440 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:26,440 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:26,440 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,441 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,443 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,443 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,443 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,443 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,444 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,444 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:26,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,445 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,445 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:26,445 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,446 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,446 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,447 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,447 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,447 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:54:26,448 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,448 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,449 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,449 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,449 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,449 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,461 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/14b8831394354804b6375c8e87646555 2023-02-09 22:54:26,474 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e 2023-02-09 22:54:26,478 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/14b8831394354804b6375c8e87646555 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555 2023-02-09 22:54:26,483 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:26,484 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e 2023-02-09 22:54:26,489 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:54:26,490 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 43ms, sequenceid=6, compaction requested=false 2023-02-09 22:54:26,490 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:26,491 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:26,491 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,491 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:26,491 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:26,491 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,491 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,491 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,493 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,493 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,494 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,494 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:26,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@475868c5[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:26,496 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:26,496 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,497 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,497 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,497 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:26,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:26,498 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:26,500 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,500 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,500 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,500 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,501 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,501 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,501 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,501 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,501 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,501 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:26,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:26,504 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:26,504 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:26,504 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:26,504 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,504 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:26,506 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,506 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,506 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,506 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,506 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,506 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,507 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,507 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,507 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,508 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,509 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,510 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,510 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,518 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:26,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:26,519 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,519 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,519 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:26,519 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:26,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,519 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,519 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,519 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,520 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,520 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,520 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,520 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,520 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,520 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,619 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:26,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:26,634 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:26,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:26,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,635 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:26,635 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,635 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:26,635 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:26,636 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,636 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,637 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,637 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,637 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,637 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,638 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,638 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,639 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:26,639 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,639 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,639 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:26,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:26,640 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:26,640 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,640 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:26,640 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:26,644 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,644 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,644 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,644 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,644 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,644 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,659 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/da437f609b9d4ec89e81116457898452 2023-02-09 22:54:26,671 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/633b539d378f4fe390dd70261b1259ae 2023-02-09 22:54:26,676 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/da437f609b9d4ec89e81116457898452 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452 2023-02-09 22:54:26,680 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:26,681 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/633b539d378f4fe390dd70261b1259ae as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae 2023-02-09 22:54:26,686 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:54:26,686 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 46ms, sequenceid=19, compaction requested=false 2023-02-09 22:54:26,687 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:26,687 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,687 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:26,687 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:26,687 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,687 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,687 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,688 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,688 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,689 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,689 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77b48061[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:26,691 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:26,691 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:26,693 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,693 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:26,695 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,695 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,695 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,695 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,696 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,696 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,696 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,696 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,696 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,696 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:26,699 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:26,699 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:26,699 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:26,699 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,699 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:26,701 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,701 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,701 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,701 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,702 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,702 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,704 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,704 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,704 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,712 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,712 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:26,713 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:26,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,713 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,714 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,714 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,714 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,714 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,814 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:26,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:26,826 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:26,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:26,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,827 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:26,827 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,827 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:26,827 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:26,828 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,828 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,829 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,829 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,829 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,829 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,830 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,830 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:26,831 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,832 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,832 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:26,832 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:26,832 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,832 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:26,832 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:26,833 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,833 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,833 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,834 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,834 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,834 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,840 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/c7084c9e9c8844aeb20b11e804b6632e 2023-02-09 22:54:26,851 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/cce3531263b149eabd3b67f15e43e974 2023-02-09 22:54:26,855 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/c7084c9e9c8844aeb20b11e804b6632e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e 2023-02-09 22:54:26,860 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:26,860 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/cce3531263b149eabd3b67f15e43e974 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974 2023-02-09 22:54:26,864 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:54:26,865 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 33ms, sequenceid=32, compaction requested=true 2023-02-09 22:54:26,865 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:26,865 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:26,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:26,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:26,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:26,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:26,872 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,872 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,872 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,872 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:26,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cd90a40[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:26,874 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:26,874 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,875 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,875 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,875 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,875 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,875 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:26,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:26,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:26,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:26,876 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:26,877 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,877 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,877 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,877 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,877 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,877 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,878 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:26,878 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,878 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,878 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:26,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:26,881 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:26,881 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:26,881 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:26,881 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,881 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:26,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,885 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,885 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,886 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,887 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,888 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,888 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,891 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:26,891 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:26,891 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,890 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,891 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,891 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,891 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:26,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,891 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:26,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:26,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:26,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:26,991 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:26,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:27,005 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:27,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:27,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,007 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,007 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,007 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:27,007 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:27,007 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,007 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,009 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,009 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,009 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,009 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,009 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,009 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,010 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,010 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,011 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,011 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:27,011 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,011 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,011 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:27,013 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,019 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,019 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,019 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,019 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,019 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,021 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13 2023-02-09 22:54:27,033 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/3d3566cd73f44e08aeda02924a350b22 2023-02-09 22:54:27,037 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13 2023-02-09 22:54:27,040 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:27,041 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/3d3566cd73f44e08aeda02924a350b22 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/3d3566cd73f44e08aeda02924a350b22 2023-02-09 22:54:27,044 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/3d3566cd73f44e08aeda02924a350b22, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:54:27,045 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 34ms, sequenceid=45, compaction requested=true 2023-02-09 22:54:27,045 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:27,045 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,046 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:27,046 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:27,046 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,046 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,046 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,048 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,048 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,048 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,048 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d50c213[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:27,050 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:27,050 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,051 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,051 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:27,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,051 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,052 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,052 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:27,053 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,053 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,053 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,055 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,055 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,056 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,056 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,056 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,056 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:27,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:27,059 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:27,059 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:27,059 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:27,059 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,059 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:27,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,065 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,065 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,065 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,066 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,067 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,068 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,068 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,072 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,072 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,072 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:27,072 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,173 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:27,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:27,185 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:27,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:27,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,186 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,187 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,187 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:27,187 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:27,187 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,187 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,188 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,188 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,188 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,188 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,189 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,189 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,190 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,190 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,190 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,190 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,190 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,190 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,190 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:27,191 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:27,191 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,191 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:27,191 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:27,192 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,193 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,193 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,193 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,193 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,193 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,202 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/3240500dc9e74b8ea2f0d792ac08df02 2023-02-09 22:54:27,213 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/901f232854c64c43bb347262bd28af65 2023-02-09 22:54:27,219 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/3240500dc9e74b8ea2f0d792ac08df02 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3240500dc9e74b8ea2f0d792ac08df02 2023-02-09 22:54:27,223 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3240500dc9e74b8ea2f0d792ac08df02, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:27,224 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/901f232854c64c43bb347262bd28af65 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/901f232854c64c43bb347262bd28af65 2023-02-09 22:54:27,228 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/901f232854c64c43bb347262bd28af65, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:54:27,229 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 38ms, sequenceid=58, compaction requested=true 2023-02-09 22:54:27,229 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:27,229 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,229 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:27,229 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:27,229 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,229 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,229 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,232 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,232 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,232 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,232 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:27,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79ca833c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:27,234 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:27,234 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,235 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,235 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,236 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:27,235 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,236 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:27,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,236 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:27,237 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,237 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,237 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,238 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,238 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,238 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,240 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,240 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,241 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,241 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:27,244 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:27,244 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:27,244 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:27,244 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:27,244 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,244 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:27,246 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,246 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,247 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,248 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,248 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,250 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,250 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,251 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,253 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,253 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,253 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,253 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,253 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,253 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:27,253 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,253 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,254 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,254 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,254 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,253 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,254 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:27,254 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:27,253 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,254 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,254 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,255 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,255 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,255 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,255 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,256 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,354 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:27,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:27,397 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:27,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:27,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,399 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,399 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,399 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:27,399 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:27,400 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,400 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,405 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,405 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,405 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,405 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,406 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,406 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,407 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,408 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,408 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,408 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,408 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:27,408 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:27,408 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,408 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:27,408 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:54:27,409 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,410 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,410 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,410 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,410 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,410 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,421 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/8fff0a2bf7a545d09b5af4f7dca1411c 2023-02-09 22:54:27,437 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/279a57ea4f5b41889c13ff913e9ff40e 2023-02-09 22:54:27,442 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/8fff0a2bf7a545d09b5af4f7dca1411c as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/8fff0a2bf7a545d09b5af4f7dca1411c 2023-02-09 22:54:27,446 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/8fff0a2bf7a545d09b5af4f7dca1411c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:27,447 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/279a57ea4f5b41889c13ff913e9ff40e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/279a57ea4f5b41889c13ff913e9ff40e 2023-02-09 22:54:27,450 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/279a57ea4f5b41889c13ff913e9ff40e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:54:27,451 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 43ms, sequenceid=71, compaction requested=true 2023-02-09 22:54:27,451 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:27,451 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,451 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:27,451 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:27,451 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,451 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,451 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,453 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,453 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,453 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,453 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:27,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a2eb3ec[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:27,456 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:27,456 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,459 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,459 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:27,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,459 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:27,460 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,460 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,461 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:27,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:27,464 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:27,464 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:27,464 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:27,464 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,464 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:27,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,466 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,467 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,467 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,468 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,469 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,469 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,472 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,472 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,472 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,473 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,473 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:27,473 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:27,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,473 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,474 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,474 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,474 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,474 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,474 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,574 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:27,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:27,585 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:27,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:27,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,587 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:27,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:27,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,588 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,589 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,589 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,589 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,589 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:27,591 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:27,591 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,591 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:27,592 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-09 22:54:27,593 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,594 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,594 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,594 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,594 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,594 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,601 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/1f49f9006acc40f99cc3eddfc1075da1 2023-02-09 22:54:27,604 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f49f9006acc40f99cc3eddfc1075da1 2023-02-09 22:54:27,611 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6 2023-02-09 22:54:27,614 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc4e8cdc00f4d60a73f4efb421fdcc6 2023-02-09 22:54:27,614 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/1f49f9006acc40f99cc3eddfc1075da1 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1 2023-02-09 22:54:27,618 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f49f9006acc40f99cc3eddfc1075da1 2023-02-09 22:54:27,618 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 22:54:27,618 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6 2023-02-09 22:54:27,621 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc4e8cdc00f4d60a73f4efb421fdcc6 2023-02-09 22:54:27,621 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 22:54:27,622 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 31ms, sequenceid=83, compaction requested=true 2023-02-09 22:54:27,622 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:27,622 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:27,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:27,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,625 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,625 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,625 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,625 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e3137e3[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:27,627 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:27,627 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,628 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,628 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,628 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,629 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,629 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:27,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,629 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:27,630 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,630 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,630 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,630 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,634 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,634 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,634 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,634 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,635 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,635 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:27,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:27,637 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:27,637 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:27,637 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:27,637 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,637 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:27,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,639 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,639 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,640 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,641 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,641 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,641 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,644 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,644 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,644 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:27,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:27,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,745 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:27,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:27,753 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:27,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:27,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,755 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,755 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,755 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:27,755 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:27,755 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,755 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,757 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,757 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,757 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,757 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,758 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,758 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:27,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,759 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,760 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,760 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,760 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:27,760 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 22:54:27,761 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,762 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,762 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,762 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,762 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,762 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,768 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:27,772 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:27,779 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:27,782 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:27,782 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/4be921204d61416eafb214d6906fb6c8 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:27,786 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:27,786 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/4be921204d61416eafb214d6906fb6c8, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 22:54:27,786 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/b1c7c350ee5a4f17bf46124ed05fb894 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:27,790 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:27,790 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/b1c7c350ee5a4f17bf46124ed05fb894, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 22:54:27,791 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 31ms, sequenceid=91, compaction requested=true 2023-02-09 22:54:27,791 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:27,791 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,791 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:27,791 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:27,791 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,791 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,791 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,793 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,793 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,793 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,793 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:27,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@450a3a6c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:27,795 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:27,795 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,797 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,797 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,798 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,798 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:27,798 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:27,798 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:27,798 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:27,797 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,798 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:27,797 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:27,799 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,799 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,799 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,799 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,799 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,799 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,800 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:27,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,800 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,800 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:27,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:27,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:27,802 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:27,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:27,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:27,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,802 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:27,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,804 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,804 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,805 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,805 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,806 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,807 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,807 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,812 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,813 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,813 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,813 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:27,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,813 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,814 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:27,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,814 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:27,814 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,814 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:27,814 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,814 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,914 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:27,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:27,924 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:27,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:27,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,925 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:27,925 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,925 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 22:54:27,925 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 22:54:27,926 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,926 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,927 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,928 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:27,928 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,929 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,929 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,929 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,929 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,929 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,929 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 22:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:27,930 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:27,930 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:27,930 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. started... 2023-02-09 22:54:27,930 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 22:54:27,931 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,932 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,932 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,932 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,932 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:27,933 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,939 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:27,943 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:28,352 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:28,356 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:28,357 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/3eff832f182e4d3da06cb4f387ea5bfa as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:28,360 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:28,360 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3eff832f182e4d3da06cb4f387ea5bfa, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 22:54:28,361 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/a42c2610262b462db445cc9a6b72404d as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:28,365 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:28,365 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/a42c2610262b462db445cc9a6b72404d, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 22:54:28,365 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 435ms, sequenceid=99, compaction requested=true 2023-02-09 22:54:28,366 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e757cfcd8ee3a29c09867ad25a0805c2: 2023-02-09 22:54:28,366 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:28,366 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:28,366 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:28,366 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,366 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 22:54:28,366 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 22:54:28,368 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,368 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:28,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:28,369 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,369 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:28,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:28,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:28,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:28,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 22:54:28,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68f49292[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:28,370 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 22:54:28,370 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:28,372 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,372 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:28,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 22:54:28,372 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:54:28,373 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:28,373 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:28,373 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,373 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:28,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:28,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:28,373 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:28,373 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:28,373 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:28,374 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 22:54:28,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:28,374 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:28,374 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 22:54:28,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:28,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:54:28,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:54:28,377 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 22:54:28,377 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:28,377 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:28,377 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,377 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:28,379 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,379 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:28,379 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,379 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,379 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,380 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:28,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,381 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:28,381 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:28,382 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,382 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,383 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:28,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:28,385 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,385 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 22:54:28,385 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:28,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:28,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:54:28,385 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:28,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,386 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:28,486 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:28,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:28,489 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,489 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:28,490 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:28,491 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:28,491 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:28,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e757cfcd8ee3a29c09867ad25a0805c2, UNASSIGN}] 2023-02-09 22:54:28,493 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e757cfcd8ee3a29c09867ad25a0805c2, UNASSIGN 2023-02-09 22:54:28,493 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e757cfcd8ee3a29c09867ad25a0805c2, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:28,494 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675983268493"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983268493"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983268493"}]},"ts":"1675983268493"} 2023-02-09 22:54:28,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure e757cfcd8ee3a29c09867ad25a0805c2, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:28,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing e757cfcd8ee3a29c09867ad25a0805c2, disabling compactions & flushes 2023-02-09 22:54:28,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:28,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:28,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. after waiting 0 ms 2023-02-09 22:54:28,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:28,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing e757cfcd8ee3a29c09867ad25a0805c2 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-09 22:54:28,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:28,660 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:28,668 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:28,671 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:28,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf1/03a44311c64847bcb3fb9793c9b89b35 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:28,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:28,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/03a44311c64847bcb3fb9793c9b89b35, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 22:54:28,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.tmp/cf2/f0c30944faef4442be5bfa340b1abd03 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:28,679 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:28,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/f0c30944faef4442be5bfa340b1abd03, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 22:54:28,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 33ms, sequenceid=103, compaction requested=true 2023-02-09 22:54:28,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-09 22:54:28,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:28,686 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. 2023-02-09 22:54:28,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for e757cfcd8ee3a29c09867ad25a0805c2: Waiting for close lock at 1675983268647Running coprocessor pre-close hooks at 1675983268647Disabling compacts and flushes for region at 1675983268647Disabling writes for close at 1675983268647Obtaining lock to block concurrent updates at 1675983268647Preparing flush snapshotting stores in e757cfcd8ee3a29c09867ad25a0805c2 at 1675983268647Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675983268647Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2. at 1675983268648 (+1 ms)Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf1: creating writer at 1675983268649 (+1 ms)Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf1: appending metadata at 1675983268650 (+1 ms)Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf1: closing flushed file at 1675983268650Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf2: creating writer at 1675983268660 (+10 ms)Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf2: appending metadata at 1675983268661 (+1 ms)Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf2: closing flushed file at 1675983268662 (+1 ms)Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf1: reopening flushed file at 1675983268672 (+10 ms)Flushing e757cfcd8ee3a29c09867ad25a0805c2/cf2: reopening flushed file at 1675983268676 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for e757cfcd8ee3a29c09867ad25a0805c2 in 33ms, sequenceid=103, compaction requested=true at 1675983268680 (+4 ms)Writing region close event to WAL at 1675983268683 (+3 ms)Running coprocessor post-close hooks at 1675983268686 (+3 ms)Closed at 1675983268686 2023-02-09 22:54:28,688 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,688 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e757cfcd8ee3a29c09867ad25a0805c2, regionState=CLOSED 2023-02-09 22:54:28,688 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675983268688"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983268688"}]},"ts":"1675983268688"} 2023-02-09 22:54:28,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-09 22:54:28,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure e757cfcd8ee3a29c09867ad25a0805c2, server=jenkins-hbase9.apache.org,40413,1675983241505 in 194 msec 2023-02-09 22:54:28,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-09 22:54:28,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e757cfcd8ee3a29c09867ad25a0805c2, UNASSIGN in 199 msec 2023-02-09 22:54:28,705 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=e757cfcd8ee3a29c09867ad25a0805c2, threads=16 2023-02-09 22:54:28,705 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/279a57ea4f5b41889c13ff913e9ff40e for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,705 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/3d3566cd73f44e08aeda02924a350b22 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,705 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,707 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,707 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/901f232854c64c43bb347262bd28af65 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,707 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,707 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/a42c2610262b462db445cc9a6b72404d for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,707 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/b1c7c350ee5a4f17bf46124ed05fb894 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,707 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,712 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/f0c30944faef4442be5bfa340b1abd03 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,712 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/03a44311c64847bcb3fb9793c9b89b35 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,712 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,713 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,713 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3240500dc9e74b8ea2f0d792ac08df02 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,713 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3eff832f182e4d3da06cb4f387ea5bfa for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,715 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/4be921204d61416eafb214d6906fb6c8 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,721 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:28,722 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/279a57ea4f5b41889c13ff913e9ff40e, top=false 2023-02-09 22:54:28,722 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:28,724 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:28,724 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:28,725 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:28,725 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:28,728 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f49f9006acc40f99cc3eddfc1075da1 2023-02-09 22:54:28,729 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/3d3566cd73f44e08aeda02924a350b22, top=false 2023-02-09 22:54:28,730 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3240500dc9e74b8ea2f0d792ac08df02, top=false 2023-02-09 22:54:28,731 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/901f232854c64c43bb347262bd28af65, top=false 2023-02-09 22:54:28,734 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc4e8cdc00f4d60a73f4efb421fdcc6 2023-02-09 22:54:28,735 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:28,735 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/03a44311c64847bcb3fb9793c9b89b35, top=true 2023-02-09 22:54:28,735 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:28,735 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3eff832f182e4d3da06cb4f387ea5bfa, top=true 2023-02-09 22:54:28,738 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:28,739 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/b1c7c350ee5a4f17bf46124ed05fb894, top=true 2023-02-09 22:54:28,739 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:28,739 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/4be921204d61416eafb214d6906fb6c8, top=true 2023-02-09 22:54:28,739 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:28,739 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:28,740 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/a42c2610262b462db445cc9a6b72404d, top=true 2023-02-09 22:54:28,740 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/f0c30944faef4442be5bfa340b1abd03, top=true 2023-02-09 22:54:28,790 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-901f232854c64c43bb347262bd28af65 for child: 069696cc5e1a2ad5e9ee2aebcbea0d8a, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,794 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/901f232854c64c43bb347262bd28af65 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,794 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/8fff0a2bf7a545d09b5af4f7dca1411c for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,797 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/8fff0a2bf7a545d09b5af4f7dca1411c, top=false 2023-02-09 22:54:28,799 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3eff832f182e4d3da06cb4f387ea5bfa for child: c115dc831efcb6b59f98b9e48a74859b, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,799 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3eff832f182e4d3da06cb4f387ea5bfa for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,799 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,800 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-a42c2610262b462db445cc9a6b72404d for child: c115dc831efcb6b59f98b9e48a74859b, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,800 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/a42c2610262b462db445cc9a6b72404d for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,800 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,803 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13, top=false 2023-02-09 22:54:28,807 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3240500dc9e74b8ea2f0d792ac08df02 for child: 069696cc5e1a2ad5e9ee2aebcbea0d8a, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,807 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-279a57ea4f5b41889c13ff913e9ff40e for child: 069696cc5e1a2ad5e9ee2aebcbea0d8a, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,810 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3240500dc9e74b8ea2f0d792ac08df02 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,810 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,810 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/279a57ea4f5b41889c13ff913e9ff40e for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,815 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-4be921204d61416eafb214d6906fb6c8 for child: c115dc831efcb6b59f98b9e48a74859b, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,815 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/4be921204d61416eafb214d6906fb6c8 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,819 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-b1c7c350ee5a4f17bf46124ed05fb894 for child: c115dc831efcb6b59f98b9e48a74859b, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,819 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/b1c7c350ee5a4f17bf46124ed05fb894 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,823 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3d3566cd73f44e08aeda02924a350b22 for child: 069696cc5e1a2ad5e9ee2aebcbea0d8a, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,827 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-03a44311c64847bcb3fb9793c9b89b35 for child: c115dc831efcb6b59f98b9e48a74859b, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,827 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/03a44311c64847bcb3fb9793c9b89b35 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,830 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/3d3566cd73f44e08aeda02924a350b22 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,834 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-f0c30944faef4442be5bfa340b1abd03 for child: c115dc831efcb6b59f98b9e48a74859b, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,834 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/f0c30944faef4442be5bfa340b1abd03 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,835 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:28,838 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f49f9006acc40f99cc3eddfc1075da1 2023-02-09 22:54:28,846 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,846 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-8fff0a2bf7a545d09b5af4f7dca1411c for child: 069696cc5e1a2ad5e9ee2aebcbea0d8a, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,847 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/.splits/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-90c27ed48d6f4e71ba5f7b5bd68c2d13 for child: 069696cc5e1a2ad5e9ee2aebcbea0d8a, parent: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,852 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/8fff0a2bf7a545d09b5af4f7dca1411c for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,872 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,876 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,877 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc4e8cdc00f4d60a73f4efb421fdcc6 2023-02-09 22:54:28,882 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,883 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,899 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,921 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,922 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,922 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452 for region: e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:28,922 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region e757cfcd8ee3a29c09867ad25a0805c2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:54:28,956 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 22:54:28,957 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 22:54:28,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675983268959"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675983268959"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675983268959"}]},"ts":"1675983268959"} 2023-02-09 22:54:28,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983268959"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983268959"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983268959"}]},"ts":"1675983268959"} 2023-02-09 22:54:28,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983268959"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983268959"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983268959"}]},"ts":"1675983268959"} 2023-02-09 22:54:28,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=069696cc5e1a2ad5e9ee2aebcbea0d8a, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c115dc831efcb6b59f98b9e48a74859b, ASSIGN}] 2023-02-09 22:54:28,963 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c115dc831efcb6b59f98b9e48a74859b, ASSIGN 2023-02-09 22:54:28,963 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c115dc831efcb6b59f98b9e48a74859b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:54:28,964 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=069696cc5e1a2ad5e9ee2aebcbea0d8a, ASSIGN 2023-02-09 22:54:28,964 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=069696cc5e1a2ad5e9ee2aebcbea0d8a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:54:29,113 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 22:54:29,115 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=069696cc5e1a2ad5e9ee2aebcbea0d8a, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:29,115 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269115"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983269115"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983269115"}]},"ts":"1675983269115"} 2023-02-09 22:54:29,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 069696cc5e1a2ad5e9ee2aebcbea0d8a, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:29,119 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=c115dc831efcb6b59f98b9e48a74859b, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:29,119 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269119"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983269119"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983269119"}]},"ts":"1675983269119"} 2023-02-09 22:54:29,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure c115dc831efcb6b59f98b9e48a74859b, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:29,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 069696cc5e1a2ad5e9ee2aebcbea0d8a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.', STARTKEY => '', ENDKEY => '60'} 2023-02-09 22:54:29,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:29,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:29,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,273 INFO [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,273 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 2023-02-09 22:54:29,273 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 2023-02-09 22:54:29,274 INFO [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 069696cc5e1a2ad5e9ee2aebcbea0d8a columnFamilyName cf1 2023-02-09 22:54:29,285 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555-bottom 2023-02-09 22:54:29,288 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,289 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1-bottom 2023-02-09 22:54:29,293 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e-bottom 2023-02-09 22:54:29,297 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452-bottom 2023-02-09 22:54:29,300 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3240500dc9e74b8ea2f0d792ac08df02 2023-02-09 22:54:29,303 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-8fff0a2bf7a545d09b5af4f7dca1411c 2023-02-09 22:54:29,306 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-90c27ed48d6f4e71ba5f7b5bd68c2d13 2023-02-09 22:54:29,306 INFO [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(310): Store=069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:29,306 INFO [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,307 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 2023-02-09 22:54:29,307 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 2023-02-09 22:54:29,307 INFO [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 069696cc5e1a2ad5e9ee2aebcbea0d8a columnFamilyName cf2 2023-02-09 22:54:29,316 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae-bottom 2023-02-09 22:54:29,320 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,321 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6-bottom 2023-02-09 22:54:29,324 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e-bottom 2023-02-09 22:54:29,328 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974-bottom 2023-02-09 22:54:29,330 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-279a57ea4f5b41889c13ff913e9ff40e 2023-02-09 22:54:29,333 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3d3566cd73f44e08aeda02924a350b22 2023-02-09 22:54:29,336 DEBUG [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-901f232854c64c43bb347262bd28af65 2023-02-09 22:54:29,336 INFO [StoreOpener-069696cc5e1a2ad5e9ee2aebcbea0d8a-1] regionserver.HStore(310): Store=069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:29,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:29,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 069696cc5e1a2ad5e9ee2aebcbea0d8a; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61471050, jitterRate=-0.08400997519493103}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:29,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 069696cc5e1a2ad5e9ee2aebcbea0d8a: Running coprocessor pre-open hook at 1675983269272Writing region info on filesystem at 1675983269272Initializing all the Stores at 1675983269272Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983269272Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983269272Cleaning up temporary data from old regions at 1675983269338 (+66 ms)Cleaning up detritus from prior splits at 1675983269338Running coprocessor post-open hooks at 1675983269341 (+3 ms)Region opened successfully at 1675983269341 2023-02-09 22:54:29,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a., pid=77, masterSystemTime=1675983269268 2023-02-09 22:54:29,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:29,343 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:29,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:29,343 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,343 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:29,343 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,344 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1912): 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 is initiating minor compaction (all files) 2023-02-09 22:54:29,343 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1912): 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 is initiating minor compaction (all files) 2023-02-09 22:54:29,344 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,344 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,344 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-90c27ed48d6f4e71ba5f7b5bd68c2d13, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3240500dc9e74b8ea2f0d792ac08df02, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-8fff0a2bf7a545d09b5af4f7dca1411c, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp, totalSize=35.0 K 2023-02-09 22:54:29,344 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3d3566cd73f44e08aeda02924a350b22, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-901f232854c64c43bb347262bd28af65, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-279a57ea4f5b41889c13ff913e9ff40e, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp, totalSize=35.0 K 2023-02-09 22:54:29,345 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983266433 2023-02-09 22:54:29,345 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983266433 2023-02-09 22:54:29,345 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983266621 2023-02-09 22:54:29,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,346 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983266621 2023-02-09 22:54:29,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c115dc831efcb6b59f98b9e48a74859b, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.', STARTKEY => '60', ENDKEY => ''} 2023-02-09 22:54:29,346 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983266815 2023-02-09 22:54:29,346 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=069696cc5e1a2ad5e9ee2aebcbea0d8a, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:29,346 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983266815 2023-02-09 22:54:29,346 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269346"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983269346"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983269346"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983269346"}]},"ts":"1675983269346"} 2023-02-09 22:54:29,346 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-90c27ed48d6f4e71ba5f7b5bd68c2d13, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983266993 2023-02-09 22:54:29,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:29,346 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3d3566cd73f44e08aeda02924a350b22, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983266993 2023-02-09 22:54:29,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:29,347 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3240500dc9e74b8ea2f0d792ac08df02, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983267174 2023-02-09 22:54:29,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,347 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-901f232854c64c43bb347262bd28af65, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983267174 2023-02-09 22:54:29,347 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-8fff0a2bf7a545d09b5af4f7dca1411c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983267356 2023-02-09 22:54:29,347 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-279a57ea4f5b41889c13ff913e9ff40e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983267356 2023-02-09 22:54:29,348 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983267575 2023-02-09 22:54:29,348 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983267575 2023-02-09 22:54:29,359 INFO [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-09 22:54:29,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 069696cc5e1a2ad5e9ee2aebcbea0d8a, server=jenkins-hbase9.apache.org,40413,1675983241505 in 231 msec 2023-02-09 22:54:29,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=069696cc5e1a2ad5e9ee2aebcbea0d8a, ASSIGN in 397 msec 2023-02-09 22:54:29,361 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1 2023-02-09 22:54:29,361 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1 2023-02-09 22:54:29,362 INFO [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c115dc831efcb6b59f98b9e48a74859b columnFamilyName cf1 2023-02-09 22:54:29,372 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555-top 2023-02-09 22:54:29,372 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): 069696cc5e1a2ad5e9ee2aebcbea0d8a#cf1#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,373 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): 069696cc5e1a2ad5e9ee2aebcbea0d8a#cf2#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,375 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,375 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1-top 2023-02-09 22:54:29,386 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e-top 2023-02-09 22:54:29,390 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp/cf2/cf573e0e4d3141f28f4e2a801f1dfacd as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cf573e0e4d3141f28f4e2a801f1dfacd 2023-02-09 22:54:29,390 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452-top 2023-02-09 22:54:29,391 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp/cf1/8a9c6b172cdc4be6baf473883e2e4cb0 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/8a9c6b172cdc4be6baf473883e2e4cb0 2023-02-09 22:54:29,396 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:29,396 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:29,398 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 of 069696cc5e1a2ad5e9ee2aebcbea0d8a into cf573e0e4d3141f28f4e2a801f1dfacd(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,398 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 069696cc5e1a2ad5e9ee2aebcbea0d8a: 2023-02-09 22:54:29,398 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a., storeName=069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2, priority=9, startTime=1675983269343; duration=0sec 2023-02-09 22:54:29,398 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:29,400 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:29,400 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:29,403 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:29,403 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:29,403 INFO [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(310): Store=c115dc831efcb6b59f98b9e48a74859b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:29,404 INFO [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,404 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 of 069696cc5e1a2ad5e9ee2aebcbea0d8a into 8a9c6b172cdc4be6baf473883e2e4cb0(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,404 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 069696cc5e1a2ad5e9ee2aebcbea0d8a: 2023-02-09 22:54:29,404 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a., storeName=069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1, priority=9, startTime=1675983269342; duration=0sec 2023-02-09 22:54:29,404 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2 2023-02-09 22:54:29,404 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:29,404 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2 2023-02-09 22:54:29,405 INFO [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c115dc831efcb6b59f98b9e48a74859b columnFamilyName cf2 2023-02-09 22:54:29,413 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae-top 2023-02-09 22:54:29,416 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,416 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6-top 2023-02-09 22:54:29,419 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e-top 2023-02-09 22:54:29,423 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974-top 2023-02-09 22:54:29,425 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:29,425 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:29,428 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:29,428 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:29,431 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:29,432 DEBUG [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:29,432 INFO [StoreOpener-c115dc831efcb6b59f98b9e48a74859b-1] regionserver.HStore(310): Store=c115dc831efcb6b59f98b9e48a74859b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:29,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:29,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened c115dc831efcb6b59f98b9e48a74859b; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59593995, jitterRate=-0.11198027431964874}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:29,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for c115dc831efcb6b59f98b9e48a74859b: Running coprocessor pre-open hook at 1675983269347Writing region info on filesystem at 1675983269347Initializing all the Stores at 1675983269358 (+11 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983269359 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983269359Cleaning up temporary data from old regions at 1675983269434 (+75 ms)Cleaning up detritus from prior splits at 1675983269434Running coprocessor post-open hooks at 1675983269437 (+3 ms)Region opened successfully at 1675983269437 2023-02-09 22:54:29,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b., pid=78, masterSystemTime=1675983269268 2023-02-09 22:54:29,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:29,439 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:29,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:29,439 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:54:29,439 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,439 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,439 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1912): c115dc831efcb6b59f98b9e48a74859b/cf1 is initiating minor compaction (all files) 2023-02-09 22:54:29,439 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1912): c115dc831efcb6b59f98b9e48a74859b/cf2 is initiating minor compaction (all files) 2023-02-09 22:54:29,439 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c115dc831efcb6b59f98b9e48a74859b/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,439 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c115dc831efcb6b59f98b9e48a74859b/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,439 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-4be921204d61416eafb214d6906fb6c8, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3eff832f182e4d3da06cb4f387ea5bfa, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-03a44311c64847bcb3fb9793c9b89b35] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp, totalSize=34.8 K 2023-02-09 22:54:29,439 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-b1c7c350ee5a4f17bf46124ed05fb894, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-a42c2610262b462db445cc9a6b72404d, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-f0c30944faef4442be5bfa340b1abd03] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp, totalSize=34.8 K 2023-02-09 22:54:29,440 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983266433 2023-02-09 22:54:29,440 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983266433 2023-02-09 22:54:29,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,441 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983266621 2023-02-09 22:54:29,441 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983266621 2023-02-09 22:54:29,441 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=c115dc831efcb6b59f98b9e48a74859b, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:29,441 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269441"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983269441"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983269441"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983269441"}]},"ts":"1675983269441"} 2023-02-09 22:54:29,442 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983266815 2023-02-09 22:54:29,442 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983266815 2023-02-09 22:54:29,442 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675983267575 2023-02-09 22:54:29,442 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675983267575 2023-02-09 22:54:29,443 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-4be921204d61416eafb214d6906fb6c8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,443 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-b1c7c350ee5a4f17bf46124ed05fb894, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,443 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3eff832f182e4d3da06cb4f387ea5bfa, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,443 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-a42c2610262b462db445cc9a6b72404d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,444 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-03a44311c64847bcb3fb9793c9b89b35, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,444 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-f0c30944faef4442be5bfa340b1abd03, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-09 22:54:29,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure c115dc831efcb6b59f98b9e48a74859b, server=jenkins-hbase9.apache.org,40413,1675983241505 in 323 msec 2023-02-09 22:54:29,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-09 22:54:29,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c115dc831efcb6b59f98b9e48a74859b, ASSIGN in 484 msec 2023-02-09 22:54:29,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b in 959 msec 2023-02-09 22:54:29,463 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c115dc831efcb6b59f98b9e48a74859b#cf1#compaction#75 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,463 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): c115dc831efcb6b59f98b9e48a74859b#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,498 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e7480732fea642f4a315cdb9f013488f: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,498 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for cf4c7587e36f45e4a0e03ae96f666933: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,498 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp/cf2/e7480732fea642f4a315cdb9f013488f as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/e7480732fea642f4a315cdb9f013488f 2023-02-09 22:54:29,499 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp/cf1/cf4c7587e36f45e4a0e03ae96f666933 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/cf4c7587e36f45e4a0e03ae96f666933 2023-02-09 22:54:29,504 DEBUG [Listener at localhost/44743] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 22:54:29,508 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e7480732fea642f4a315cdb9f013488f: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,508 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:60428, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 22:54:29,509 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,509 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for cf4c7587e36f45e4a0e03ae96f666933: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,509 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c115dc831efcb6b59f98b9e48a74859b/cf2 of c115dc831efcb6b59f98b9e48a74859b into e7480732fea642f4a315cdb9f013488f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,509 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c115dc831efcb6b59f98b9e48a74859b: 2023-02-09 22:54:29,509 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b., storeName=c115dc831efcb6b59f98b9e48a74859b/cf2, priority=9, startTime=1675983269439; duration=0sec 2023-02-09 22:54:29,510 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 22:54:29,510 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] regionserver.HStore(1912): 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 is initiating major compaction (all files) 2023-02-09 22:54:29,510 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c115dc831efcb6b59f98b9e48a74859b/cf1 of c115dc831efcb6b59f98b9e48a74859b into cf4c7587e36f45e4a0e03ae96f666933(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,510 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4d953486; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:29,510 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c115dc831efcb6b59f98b9e48a74859b: 2023-02-09 22:54:29,510 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b., storeName=c115dc831efcb6b59f98b9e48a74859b/cf1, priority=9, startTime=1675983269438; duration=0sec 2023-02-09 22:54:29,510 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 22:54:29,510 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] regionserver.HStore(1912): 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 is initiating major compaction (all files) 2023-02-09 22:54:29,511 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@232f7ff9; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 22:54:29,511 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 22:54:29,511 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:29,511 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,511 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,511 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/8a9c6b172cdc4be6baf473883e2e4cb0] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp, totalSize=6.8 K 2023-02-09 22:54:29,511 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cf573e0e4d3141f28f4e2a801f1dfacd] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp, totalSize=6.8 K 2023-02-09 22:54:29,511 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 8a9c6b172cdc4be6baf473883e2e4cb0, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983266433 2023-02-09 22:54:29,511 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting cf573e0e4d3141f28f4e2a801f1dfacd, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983266433 2023-02-09 22:54:29,513 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 22:54:29,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] regionserver.HStore(1912): c115dc831efcb6b59f98b9e48a74859b/cf1 is initiating major compaction (all files) 2023-02-09 22:54:29,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5dcdc1cc; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:29,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 22:54:29,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] regionserver.HStore(1912): c115dc831efcb6b59f98b9e48a74859b/cf2 is initiating major compaction (all files) 2023-02-09 22:54:29,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6a738793; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 22:54:29,514 INFO [Listener at localhost/44743] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 22:54:29,516 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): 069696cc5e1a2ad5e9ee2aebcbea0d8a#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,516 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): 069696cc5e1a2ad5e9ee2aebcbea0d8a#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,530 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. is not mergeable because it has references 2023-02-09 22:54:29,531 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. is not mergeable because it has references 2023-02-09 22:54:29,535 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp/cf2/d3f1249aa2d045e9bd23ab2db90acee8 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/d3f1249aa2d045e9bd23ab2db90acee8 2023-02-09 22:54:29,536 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/.tmp/cf1/faa061298114472280be29e2686fa048 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/faa061298114472280be29e2686fa048 2023-02-09 22:54:29,540 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2 of 069696cc5e1a2ad5e9ee2aebcbea0d8a into d3f1249aa2d045e9bd23ab2db90acee8(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,540 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 069696cc5e1a2ad5e9ee2aebcbea0d8a: 2023-02-09 22:54:29,540 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1 of 069696cc5e1a2ad5e9ee2aebcbea0d8a into faa061298114472280be29e2686fa048(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,540 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 069696cc5e1a2ad5e9ee2aebcbea0d8a: 2023-02-09 22:54:29,541 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675983269510; duration=0sec 2023-02-09 22:54:29,541 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675983269510; duration=0sec 2023-02-09 22:54:29,541 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 22:54:29,541 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:54:29,541 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c115dc831efcb6b59f98b9e48a74859b/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,541 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c115dc831efcb6b59f98b9e48a74859b/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,541 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/e7480732fea642f4a315cdb9f013488f] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp, totalSize=5.0 K 2023-02-09 22:54:29,541 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/cf4c7587e36f45e4a0e03ae96f666933] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp, totalSize=5.0 K 2023-02-09 22:54:29,542 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting e7480732fea642f4a315cdb9f013488f, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,542 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting cf4c7587e36f45e4a0e03ae96f666933, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 22:54:29,545 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): c115dc831efcb6b59f98b9e48a74859b#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,547 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): c115dc831efcb6b59f98b9e48a74859b#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:29,557 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8c48d4b3619940f49b7879de72031135: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,558 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6c023fb8b1fb4055827cb1161479ec2b: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,558 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp/cf1/8c48d4b3619940f49b7879de72031135 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/8c48d4b3619940f49b7879de72031135 2023-02-09 22:54:29,558 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/.tmp/cf2/6c023fb8b1fb4055827cb1161479ec2b as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6c023fb8b1fb4055827cb1161479ec2b 2023-02-09 22:54:29,561 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6c023fb8b1fb4055827cb1161479ec2b: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,562 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8c48d4b3619940f49b7879de72031135: NONE, but ROW specified in column family configuration 2023-02-09 22:54:29,562 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c115dc831efcb6b59f98b9e48a74859b/cf1 of c115dc831efcb6b59f98b9e48a74859b into 8c48d4b3619940f49b7879de72031135(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,562 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c115dc831efcb6b59f98b9e48a74859b/cf2 of c115dc831efcb6b59f98b9e48a74859b into 6c023fb8b1fb4055827cb1161479ec2b(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:29,562 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c115dc831efcb6b59f98b9e48a74859b: 2023-02-09 22:54:29,563 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c115dc831efcb6b59f98b9e48a74859b: 2023-02-09 22:54:29,563 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675983269513; duration=0sec 2023-02-09 22:54:29,563 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675983269513; duration=0sec 2023-02-09 22:54:29,563 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:29,563 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:29,638 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40413] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. is not mergeable because it has references 2023-02-09 22:54:29,639 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40413] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. is not mergeable because it has references 2023-02-09 22:54:29,644 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-09 22:54:29,646 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:29,647 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,648 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:54:29,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983269655"}]},"ts":"1675983269655"} 2023-02-09 22:54:29,656 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-09 22:54:29,658 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-09 22:54:29,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=069696cc5e1a2ad5e9ee2aebcbea0d8a, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c115dc831efcb6b59f98b9e48a74859b, UNASSIGN}] 2023-02-09 22:54:29,659 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c115dc831efcb6b59f98b9e48a74859b, UNASSIGN 2023-02-09 22:54:29,660 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=c115dc831efcb6b59f98b9e48a74859b, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:29,660 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269660"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983269660"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983269660"}]},"ts":"1675983269660"} 2023-02-09 22:54:29,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure c115dc831efcb6b59f98b9e48a74859b, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:29,663 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=069696cc5e1a2ad5e9ee2aebcbea0d8a, UNASSIGN 2023-02-09 22:54:29,664 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=069696cc5e1a2ad5e9ee2aebcbea0d8a, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:54:29,664 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269664"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983269664"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983269664"}]},"ts":"1675983269664"} 2023-02-09 22:54:29,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 069696cc5e1a2ad5e9ee2aebcbea0d8a, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:54:29,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:54:29,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 069696cc5e1a2ad5e9ee2aebcbea0d8a, disabling compactions & flushes 2023-02-09 22:54:29,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. after waiting 0 ms 2023-02-09 22:54:29,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,814 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-90c27ed48d6f4e71ba5f7b5bd68c2d13, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3240500dc9e74b8ea2f0d792ac08df02, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-8fff0a2bf7a545d09b5af4f7dca1411c, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/8a9c6b172cdc4be6baf473883e2e4cb0, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1-bottom] to archive 2023-02-09 22:54:29,814 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:29,816 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,817 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,818 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,819 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-90c27ed48d6f4e71ba5f7b5bd68c2d13 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-90c27ed48d6f4e71ba5f7b5bd68c2d13 2023-02-09 22:54:29,820 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3240500dc9e74b8ea2f0d792ac08df02 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3240500dc9e74b8ea2f0d792ac08df02 2023-02-09 22:54:29,821 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-8fff0a2bf7a545d09b5af4f7dca1411c to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-8fff0a2bf7a545d09b5af4f7dca1411c 2023-02-09 22:54:29,822 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/8a9c6b172cdc4be6baf473883e2e4cb0 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/8a9c6b172cdc4be6baf473883e2e4cb0 2023-02-09 22:54:29,822 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,824 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3d3566cd73f44e08aeda02924a350b22, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-901f232854c64c43bb347262bd28af65, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-279a57ea4f5b41889c13ff913e9ff40e, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cf573e0e4d3141f28f4e2a801f1dfacd, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6-bottom] to archive 2023-02-09 22:54:29,824 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:29,825 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,826 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,827 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,828 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3d3566cd73f44e08aeda02924a350b22 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3d3566cd73f44e08aeda02924a350b22 2023-02-09 22:54:29,829 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-901f232854c64c43bb347262bd28af65 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-901f232854c64c43bb347262bd28af65 2023-02-09 22:54:29,829 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-279a57ea4f5b41889c13ff913e9ff40e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-279a57ea4f5b41889c13ff913e9ff40e 2023-02-09 22:54:29,830 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cf573e0e4d3141f28f4e2a801f1dfacd to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/cf573e0e4d3141f28f4e2a801f1dfacd 2023-02-09 22:54:29,831 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 22:54:29,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:29,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a. 2023-02-09 22:54:29,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 069696cc5e1a2ad5e9ee2aebcbea0d8a: Waiting for close lock at 1675983269813Running coprocessor pre-close hooks at 1675983269813Disabling compacts and flushes for region at 1675983269813Disabling writes for close at 1675983269813Writing region close event to WAL at 1675983269832 (+19 ms)Running coprocessor post-close hooks at 1675983269834 (+2 ms)Closed at 1675983269834 2023-02-09 22:54:29,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing c115dc831efcb6b59f98b9e48a74859b, disabling compactions & flushes 2023-02-09 22:54:29,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. after waiting 0 ms 2023-02-09 22:54:29,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,836 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=069696cc5e1a2ad5e9ee2aebcbea0d8a, regionState=CLOSED 2023-02-09 22:54:29,836 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269836"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983269836"}]},"ts":"1675983269836"} 2023-02-09 22:54:29,837 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-4be921204d61416eafb214d6906fb6c8, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3eff832f182e4d3da06cb4f387ea5bfa, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/cf4c7587e36f45e4a0e03ae96f666933, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-03a44311c64847bcb3fb9793c9b89b35] to archive 2023-02-09 22:54:29,838 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:29,839 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/14b8831394354804b6375c8e87646555.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-09 22:54:29,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 069696cc5e1a2ad5e9ee2aebcbea0d8a, server=jenkins-hbase9.apache.org,40413,1675983241505 in 173 msec 2023-02-09 22:54:29,840 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/da437f609b9d4ec89e81116457898452.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=069696cc5e1a2ad5e9ee2aebcbea0d8a, UNASSIGN in 181 msec 2023-02-09 22:54:29,841 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/c7084c9e9c8844aeb20b11e804b6632e.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,842 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/1f49f9006acc40f99cc3eddfc1075da1.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,843 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-4be921204d61416eafb214d6906fb6c8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:29,844 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3eff832f182e4d3da06cb4f387ea5bfa to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:29,844 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/cf4c7587e36f45e4a0e03ae96f666933 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/cf4c7587e36f45e4a0e03ae96f666933 2023-02-09 22:54:29,845 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-03a44311c64847bcb3fb9793c9b89b35 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:29,847 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-b1c7c350ee5a4f17bf46124ed05fb894, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-a42c2610262b462db445cc9a6b72404d, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/e7480732fea642f4a315cdb9f013488f, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-f0c30944faef4442be5bfa340b1abd03] to archive 2023-02-09 22:54:29,847 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:29,848 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,849 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/633b539d378f4fe390dd70261b1259ae.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,850 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/cce3531263b149eabd3b67f15e43e974.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,851 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6.e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,852 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-b1c7c350ee5a4f17bf46124ed05fb894 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:29,852 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-a42c2610262b462db445cc9a6b72404d to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:29,853 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/e7480732fea642f4a315cdb9f013488f to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/e7480732fea642f4a315cdb9f013488f 2023-02-09 22:54:29,854 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-f0c30944faef4442be5bfa340b1abd03 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/testSplitTableRegionDeletedRowsDaughter=e757cfcd8ee3a29c09867ad25a0805c2-f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:29,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 22:54:29,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:29,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b. 2023-02-09 22:54:29,858 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for c115dc831efcb6b59f98b9e48a74859b: Waiting for close lock at 1675983269836Running coprocessor pre-close hooks at 1675983269836Disabling compacts and flushes for region at 1675983269836Disabling writes for close at 1675983269836Writing region close event to WAL at 1675983269855 (+19 ms)Running coprocessor post-close hooks at 1675983269858 (+3 ms)Closed at 1675983269858 2023-02-09 22:54:29,859 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,860 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=c115dc831efcb6b59f98b9e48a74859b, regionState=CLOSED 2023-02-09 22:54:29,860 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675983269860"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983269860"}]},"ts":"1675983269860"} 2023-02-09 22:54:29,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-09 22:54:29,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure c115dc831efcb6b59f98b9e48a74859b, server=jenkins-hbase9.apache.org,40413,1675983241505 in 200 msec 2023-02-09 22:54:29,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-09 22:54:29,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c115dc831efcb6b59f98b9e48a74859b, UNASSIGN in 204 msec 2023-02-09 22:54:29,864 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983269864"}]},"ts":"1675983269864"} 2023-02-09 22:54:29,865 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-09 22:54:29,867 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-09 22:54:29,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 219 msec 2023-02-09 22:54:29,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:54:29,953 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-09 22:54:29,954 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,956 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,957 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 22:54:29,961 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,961 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,961 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,963 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/recovered.edits] 2023-02-09 22:54:29,963 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/recovered.edits] 2023-02-09 22:54:29,963 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/recovered.edits] 2023-02-09 22:54:29,965 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/faa061298114472280be29e2686fa048 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf1/faa061298114472280be29e2686fa048 2023-02-09 22:54:29,966 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/8c48d4b3619940f49b7879de72031135 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf1/8c48d4b3619940f49b7879de72031135 2023-02-09 22:54:29,966 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/03a44311c64847bcb3fb9793c9b89b35 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/03a44311c64847bcb3fb9793c9b89b35 2023-02-09 22:54:29,967 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/14b8831394354804b6375c8e87646555 2023-02-09 22:54:29,968 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/d3f1249aa2d045e9bd23ab2db90acee8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/cf2/d3f1249aa2d045e9bd23ab2db90acee8 2023-02-09 22:54:29,968 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6c023fb8b1fb4055827cb1161479ec2b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/cf2/6c023fb8b1fb4055827cb1161479ec2b 2023-02-09 22:54:29,968 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/1f49f9006acc40f99cc3eddfc1075da1 2023-02-09 22:54:29,970 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3240500dc9e74b8ea2f0d792ac08df02 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3240500dc9e74b8ea2f0d792ac08df02 2023-02-09 22:54:29,971 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/recovered.edits/113.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b/recovered.edits/113.seqid 2023-02-09 22:54:29,971 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3eff832f182e4d3da06cb4f387ea5bfa to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/3eff832f182e4d3da06cb4f387ea5bfa 2023-02-09 22:54:29,971 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/recovered.edits/113.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a/recovered.edits/113.seqid 2023-02-09 22:54:29,972 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:54:29,972 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/069696cc5e1a2ad5e9ee2aebcbea0d8a 2023-02-09 22:54:29,972 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/4be921204d61416eafb214d6906fb6c8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/4be921204d61416eafb214d6906fb6c8 2023-02-09 22:54:29,973 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/8fff0a2bf7a545d09b5af4f7dca1411c to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/8fff0a2bf7a545d09b5af4f7dca1411c 2023-02-09 22:54:29,975 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/90c27ed48d6f4e71ba5f7b5bd68c2d13 2023-02-09 22:54:29,976 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/c7084c9e9c8844aeb20b11e804b6632e 2023-02-09 22:54:29,977 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf1/da437f609b9d4ec89e81116457898452 2023-02-09 22:54:29,979 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/279a57ea4f5b41889c13ff913e9ff40e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/279a57ea4f5b41889c13ff913e9ff40e 2023-02-09 22:54:29,980 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/3d3566cd73f44e08aeda02924a350b22 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/3d3566cd73f44e08aeda02924a350b22 2023-02-09 22:54:29,982 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/633b539d378f4fe390dd70261b1259ae 2023-02-09 22:54:29,983 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/6bc4e8cdc00f4d60a73f4efb421fdcc6 2023-02-09 22:54:29,984 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/72e776f1bc4647f3bce7ab7ae22b0a5e 2023-02-09 22:54:29,985 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/901f232854c64c43bb347262bd28af65 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/901f232854c64c43bb347262bd28af65 2023-02-09 22:54:29,986 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/a42c2610262b462db445cc9a6b72404d to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/a42c2610262b462db445cc9a6b72404d 2023-02-09 22:54:29,987 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/b1c7c350ee5a4f17bf46124ed05fb894 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/b1c7c350ee5a4f17bf46124ed05fb894 2023-02-09 22:54:29,988 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/cce3531263b149eabd3b67f15e43e974 2023-02-09 22:54:29,989 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/f0c30944faef4442be5bfa340b1abd03 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/cf2/f0c30944faef4442be5bfa340b1abd03 2023-02-09 22:54:29,991 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/recovered.edits/106.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2/recovered.edits/106.seqid 2023-02-09 22:54:29,991 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e757cfcd8ee3a29c09867ad25a0805c2 2023-02-09 22:54:29,991 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 22:54:29,993 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,995 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-09 22:54:29,996 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-09 22:54:29,997 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:29,997 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-09 22:54:29,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983269997"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:29,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983269997"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:29,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983269997"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:29,999 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:54:29,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e757cfcd8ee3a29c09867ad25a0805c2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675983265929.e757cfcd8ee3a29c09867ad25a0805c2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 069696cc5e1a2ad5e9ee2aebcbea0d8a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675983268489.069696cc5e1a2ad5e9ee2aebcbea0d8a.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => c115dc831efcb6b59f98b9e48a74859b, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675983268489.c115dc831efcb6b59f98b9e48a74859b.', STARTKEY => '60', ENDKEY => ''}] 2023-02-09 22:54:29,999 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-09 22:54:29,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983269999"}]},"ts":"9223372036854775807"} 2023-02-09 22:54:30,000 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-09 22:54:30,003 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:54:30,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 49 msec 2023-02-09 22:54:30,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 22:54:30,058 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-09 22:54:30,073 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=412 (was 407) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:51130 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:47020 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:51192 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:51092 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:52610 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:52902 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:52894 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:47476 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:47444 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:47484 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:47460 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:51088 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:51116 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:51068 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:52846 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_293057512_15 at /127.0.0.1:51104 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=869 (was 865) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=193 (was 210), ProcessCount=168 (was 168), AvailableMemoryMB=10854 (was 10926) 2023-02-09 22:54:30,088 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=412, OpenFileDescriptor=869, MaxFileDescriptor=60000, SystemLoadAverage=193, ProcessCount=168, AvailableMemoryMB=10853 2023-02-09 22:54:30,088 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:54:30,091 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:54:30,091 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:54:30,093 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:54:30,094 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:30,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:30,096 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,096 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b empty. 2023-02-09 22:54:30,097 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,097 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 22:54:30,110 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:30,111 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => a5f8afc611ffecae749b3b56718b0e8b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:54:30,118 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:30,118 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing a5f8afc611ffecae749b3b56718b0e8b, disabling compactions & flushes 2023-02-09 22:54:30,118 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,118 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,118 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. after waiting 0 ms 2023-02-09 22:54:30,118 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,118 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,118 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for a5f8afc611ffecae749b3b56718b0e8b: Waiting for close lock at 1675983270118Disabling compacts and flushes for region at 1675983270118Disabling writes for close at 1675983270118Writing region close event to WAL at 1675983270118Closed at 1675983270118 2023-02-09 22:54:30,119 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:30,120 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983270120"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983270120"}]},"ts":"1675983270120"} 2023-02-09 22:54:30,121 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:30,122 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:30,122 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983270122"}]},"ts":"1675983270122"} 2023-02-09 22:54:30,123 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-09 22:54:30,125 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:30,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:30,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:30,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:30,126 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:54:30,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, ASSIGN}] 2023-02-09 22:54:30,127 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, ASSIGN 2023-02-09 22:54:30,128 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=false 2023-02-09 22:54:30,278 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:30,281 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:30,281 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983270281"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983270281"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983270281"}]},"ts":"1675983270281"} 2023-02-09 22:54:30,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:30,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a5f8afc611ffecae749b3b56718b0e8b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:30,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:30,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:30,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,442 INFO [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,444 DEBUG [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/cf1 2023-02-09 22:54:30,444 DEBUG [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/cf1 2023-02-09 22:54:30,444 INFO [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5f8afc611ffecae749b3b56718b0e8b columnFamilyName cf1 2023-02-09 22:54:30,445 INFO [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] regionserver.HStore(310): Store=a5f8afc611ffecae749b3b56718b0e8b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:30,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:30,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened a5f8afc611ffecae749b3b56718b0e8b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59423941, jitterRate=-0.11451427638530731}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:30,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for a5f8afc611ffecae749b3b56718b0e8b: Running coprocessor pre-open hook at 1675983270441Writing region info on filesystem at 1675983270441Initializing all the Stores at 1675983270441Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983270441Cleaning up temporary data from old regions at 1675983270446 (+5 ms)Cleaning up detritus from prior splits at 1675983270446Running coprocessor post-open hooks at 1675983270451 (+5 ms)Region opened successfully at 1675983270451 2023-02-09 22:54:30,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b., pid=87, masterSystemTime=1675983270433 2023-02-09 22:54:30,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,453 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:30,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983270453"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983270453"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983270453"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983270453"}]},"ts":"1675983270453"} 2023-02-09 22:54:30,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-09 22:54:30,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473 in 173 msec 2023-02-09 22:54:30,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-09 22:54:30,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, ASSIGN in 330 msec 2023-02-09 22:54:30,458 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:30,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983270458"}]},"ts":"1675983270458"} 2023-02-09 22:54:30,460 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-09 22:54:30,468 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:30,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 376 msec 2023-02-09 22:54:30,601 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$12(2441): Client=jenkins//172.31.2.10 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 22:54:30,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:54:30,623 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 22:54:30,623 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:30,624 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 22:54:30,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-09 22:54:30,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, REOPEN/MOVE}] 2023-02-09 22:54:30,639 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, REOPEN/MOVE 2023-02-09 22:54:30,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:30,640 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983270640"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983270640"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983270640"}]},"ts":"1675983270640"} 2023-02-09 22:54:30,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:30,793 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,794 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing a5f8afc611ffecae749b3b56718b0e8b, disabling compactions & flushes 2023-02-09 22:54:30,794 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,794 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,794 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. after waiting 0 ms 2023-02-09 22:54:30,794 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 22:54:30,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:54:30,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:30,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for a5f8afc611ffecae749b3b56718b0e8b: Waiting for close lock at 1675983270793Running coprocessor pre-close hooks at 1675983270793Disabling compacts and flushes for region at 1675983270793Disabling writes for close at 1675983270794 (+1 ms)Writing region close event to WAL at 1675983270794Running coprocessor post-close hooks at 1675983270798 (+4 ms)Closed at 1675983270798 2023-02-09 22:54:30,798 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a5f8afc611ffecae749b3b56718b0e8b to self. 2023-02-09 22:54:30,799 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:30,799 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=CLOSED 2023-02-09 22:54:30,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983270799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983270799"}]},"ts":"1675983270799"} 2023-02-09 22:54:30,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-09 22:54:30,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473 in 160 msec 2023-02-09 22:54:30,803 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=true 2023-02-09 22:54:30,953 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:54:30,953 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:30,953 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983270953"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983270953"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983270953"}]},"ts":"1675983270953"} 2023-02-09 22:54:30,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:54:31,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:31,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a5f8afc611ffecae749b3b56718b0e8b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:31,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:31,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:31,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:31,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:31,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:31,110 INFO [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:31,111 DEBUG [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/cf1 2023-02-09 22:54:31,111 DEBUG [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/cf1 2023-02-09 22:54:31,112 INFO [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5f8afc611ffecae749b3b56718b0e8b columnFamilyName cf1 2023-02-09 22:54:31,112 INFO [StoreOpener-a5f8afc611ffecae749b3b56718b0e8b-1] regionserver.HStore(310): Store=a5f8afc611ffecae749b3b56718b0e8b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:31,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:31,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:31,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:54:31,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened a5f8afc611ffecae749b3b56718b0e8b; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68093096, jitterRate=0.014666199684143066}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:31,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for a5f8afc611ffecae749b3b56718b0e8b: Running coprocessor pre-open hook at 1675983271110Writing region info on filesystem at 1675983271110Initializing all the Stores at 1675983271110Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983271110Cleaning up temporary data from old regions at 1675983271114 (+4 ms)Cleaning up detritus from prior splits at 1675983271114Running coprocessor post-open hooks at 1675983271116 (+2 ms)Region opened successfully at 1675983271116 2023-02-09 22:54:31,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b., pid=92, masterSystemTime=1675983271106 2023-02-09 22:54:31,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:31,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:54:31,119 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:54:31,119 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983271119"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983271119"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983271119"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983271119"}]},"ts":"1675983271119"} 2023-02-09 22:54:31,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-09 22:54:31,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473 in 165 msec 2023-02-09 22:54:31,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-09 22:54:31,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, REOPEN/MOVE in 484 msec 2023-02-09 22:54:31,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-09 22:54:31,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 498 msec 2023-02-09 22:54:31,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb, ASSIGN}] 2023-02-09 22:54:31,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb, ASSIGN 2023-02-09 22:54:31,129 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 22:54:31,280 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:54:31,281 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:54:31,281 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:54:31,281 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:54:31,281 DEBUG [jenkins-hbase9:44215] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-09 22:54:31,282 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=27ab93e5042ea955274af1f95b42abeb, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:54:31,282 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983271282"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675983271282"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675983271282"}]},"ts":"1675983271282"} 2023-02-09 22:54:31,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 27ab93e5042ea955274af1f95b42abeb, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:54:31,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. 2023-02-09 22:54:31,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 27ab93e5042ea955274af1f95b42abeb, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-09 22:54:31,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:54:31,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 27ab93e5042ea955274af1f95b42abeb 2023-02-09 22:54:31,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:31,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 27ab93e5042ea955274af1f95b42abeb 2023-02-09 22:54:31,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 27ab93e5042ea955274af1f95b42abeb 2023-02-09 22:54:31,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:31,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:31,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:31,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:31,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:31,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,816 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-09 22:54:32,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:32,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:33,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,188 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:34,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:34,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:35,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:36,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:37,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:38,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:39,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:40,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:41,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:42,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:43,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:44,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:45,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:46,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:47,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:48,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:49,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:50,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:51,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:52,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:53,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:54,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:55,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:56,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:57,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:58,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:54:59,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:00,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:01,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,877 DEBUG [regionserver/jenkins-hbase9:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 22:55:02,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:02,910 DEBUG [regionserver/jenkins-hbase9:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 22:55:02,936 DEBUG [regionserver/jenkins-hbase9:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 22:55:03,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:03,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:04,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:05,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:06,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:07,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:08,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:09,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:10,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:11,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:12,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:13,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:14,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:15,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:16,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:17,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:18,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:19,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:20,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:21,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:22,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:23,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:24,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:25,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:26,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:27,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:28,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:29,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:30,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:30,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:30,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:31,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:31,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:31,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:31,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,815 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:32,816 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:32,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:32,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:32,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:32,819 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting a5f8afc611ffecae749b3b56718b0e8b, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:32,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f exec-time=6 msec 2023-02-09 22:55:32,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:32,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:33,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 22:55:33,163 INFO [StoreOpener-27ab93e5042ea955274af1f95b42abeb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 27ab93e5042ea955274af1f95b42abeb 2023-02-09 22:55:33,166 DEBUG [StoreOpener-27ab93e5042ea955274af1f95b42abeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/cf1 2023-02-09 22:55:33,166 DEBUG [StoreOpener-27ab93e5042ea955274af1f95b42abeb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/cf1 2023-02-09 22:55:33,167 INFO [StoreOpener-27ab93e5042ea955274af1f95b42abeb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27ab93e5042ea955274af1f95b42abeb columnFamilyName cf1 2023-02-09 22:55:33,168 INFO [StoreOpener-27ab93e5042ea955274af1f95b42abeb-1] regionserver.HStore(310): Store=27ab93e5042ea955274af1f95b42abeb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:33,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 27ab93e5042ea955274af1f95b42abeb; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64239445, jitterRate=-0.04275767505168915}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:55:33,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 27ab93e5042ea955274af1f95b42abeb: Running coprocessor pre-open hook at 1675983271440Initializing all the Stores at 1675983333163 (+61723 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983333163Running coprocessor post-open hooks at 1675983333169 (+6 ms)Region opened successfully at 1675983333169 2023-02-09 22:55:33,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb., pid=94, masterSystemTime=1675983271437 2023-02-09 22:55:33,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. 2023-02-09 22:55:33,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. 2023-02-09 22:55:33,171 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=27ab93e5042ea955274af1f95b42abeb, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:33,171 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983333171"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1675983333171"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675983333171"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675983333171"}]},"ts":"1675983333171"} 2023-02-09 22:55:33,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-09 22:55:33,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 27ab93e5042ea955274af1f95b42abeb, server=jenkins-hbase9.apache.org,41885,1675983241428 in 1 mins, 1.889 sec 2023-02-09 22:55:33,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-09 22:55:33,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb, ASSIGN in 1 mins, 2.047 sec 2023-02-09 22:55:33,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.573 sec 2023-02-09 22:55:33,179 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675983270599","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62579,"client":"172.31.2.10:59126","queuetimems":0,"class":"HMaster"} 2023-02-09 22:55:33,179 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-09 22:55:35,067 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up 2023-02-09 22:55:35,068 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:35,071 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,072 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 22:55:35,076 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983335076"}]},"ts":"1675983335076"} 2023-02-09 22:55:35,077 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-09 22:55:35,081 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-09 22:55:35,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb, UNASSIGN}] 2023-02-09 22:55:35,082 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb, UNASSIGN 2023-02-09 22:55:35,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=27ab93e5042ea955274af1f95b42abeb, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:35,084 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983335083"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675983335083"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675983335083"}]},"ts":"1675983335083"} 2023-02-09 22:55:35,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 27ab93e5042ea955274af1f95b42abeb, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:55:35,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, UNASSIGN 2023-02-09 22:55:35,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:35,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983335088"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983335088"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983335088"}]},"ts":"1675983335088"} 2023-02-09 22:55:35,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:55:35,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 22:55:35,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 27ab93e5042ea955274af1f95b42abeb 2023-02-09 22:55:35,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 27ab93e5042ea955274af1f95b42abeb, disabling compactions & flushes 2023-02-09 22:55:35,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. 2023-02-09 22:55:35,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. 2023-02-09 22:55:35,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. after waiting 0 ms 2023-02-09 22:55:35,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. 2023-02-09 22:55:35,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:35,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb. 2023-02-09 22:55:35,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 27ab93e5042ea955274af1f95b42abeb: Waiting for close lock at 1675983335238Running coprocessor pre-close hooks at 1675983335238Disabling compacts and flushes for region at 1675983335238Disabling writes for close at 1675983335239 (+1 ms)Writing region close event to WAL at 1675983335239Running coprocessor post-close hooks at 1675983335239Closed at 1675983335240 (+1 ms) 2023-02-09 22:55:35,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 27ab93e5042ea955274af1f95b42abeb 2023-02-09 22:55:35,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=27ab93e5042ea955274af1f95b42abeb, regionState=CLOSED 2023-02-09 22:55:35,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:55:35,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983335241"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675983335241"}]},"ts":"1675983335241"} 2023-02-09 22:55:35,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing a5f8afc611ffecae749b3b56718b0e8b, disabling compactions & flushes 2023-02-09 22:55:35,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:55:35,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:55:35,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. after waiting 0 ms 2023-02-09 22:55:35,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:55:35,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-09 22:55:35,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 27ab93e5042ea955274af1f95b42abeb, server=jenkins-hbase9.apache.org,41885,1675983241428 in 158 msec 2023-02-09 22:55:35,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 22:55:35,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb, UNASSIGN in 165 msec 2023-02-09 22:55:35,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:35,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b. 2023-02-09 22:55:35,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for a5f8afc611ffecae749b3b56718b0e8b: Waiting for close lock at 1675983335242Running coprocessor pre-close hooks at 1675983335242Disabling compacts and flushes for region at 1675983335242Disabling writes for close at 1675983335242Writing region close event to WAL at 1675983335243 (+1 ms)Running coprocessor post-close hooks at 1675983335247 (+4 ms)Closed at 1675983335248 (+1 ms) 2023-02-09 22:55:35,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:55:35,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=a5f8afc611ffecae749b3b56718b0e8b, regionState=CLOSED 2023-02-09 22:55:35,249 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675983335249"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983335249"}]},"ts":"1675983335249"} 2023-02-09 22:55:35,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-09 22:55:35,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure a5f8afc611ffecae749b3b56718b0e8b, server=jenkins-hbase9.apache.org,44627,1675983241473 in 160 msec 2023-02-09 22:55:35,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-09 22:55:35,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a5f8afc611ffecae749b3b56718b0e8b, UNASSIGN in 171 msec 2023-02-09 22:55:35,254 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983335254"}]},"ts":"1675983335254"} 2023-02-09 22:55:35,255 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-09 22:55:35,256 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-09 22:55:35,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 185 msec 2023-02-09 22:55:35,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 22:55:35,378 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-09 22:55:35,378 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,381 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,381 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 22:55:35,385 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:55:35,386 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/recovered.edits] 2023-02-09 22:55:35,390 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/recovered.edits/7.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b/recovered.edits/7.seqid 2023-02-09 22:55:35,390 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/a5f8afc611ffecae749b3b56718b0e8b 2023-02-09 22:55:35,390 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 22:55:35,392 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,394 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-09 22:55:35,396 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-09 22:55:35,397 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,397 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-09 22:55:35,397 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983335397"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:35,397 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983335397"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:35,398 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-09 22:55:35,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a5f8afc611ffecae749b3b56718b0e8b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675983270092.a5f8afc611ffecae749b3b56718b0e8b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 27ab93e5042ea955274af1f95b42abeb, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675983270092_0001.27ab93e5042ea955274af1f95b42abeb.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-09 22:55:35,399 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-09 22:55:35,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983335399"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:35,400 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-09 22:55:35,402 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:35,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 24 msec 2023-02-09 22:55:35,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 22:55:35,482 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-09 22:55:35,495 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=345 (was 412), OpenFileDescriptor=655 (was 869), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 193), ProcessCount=168 (was 168), AvailableMemoryMB=10684 (was 10853) 2023-02-09 22:55:35,507 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=346, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=168, AvailableMemoryMB=10683 2023-02-09 22:55:35,507 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:35,510 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:55:35,510 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:55:35,511 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:35,512 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:35,513 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:35,514 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,515 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a empty. 2023-02-09 22:55:35,515 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,515 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 22:55:35,524 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:35,525 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => ed051d3f7e34cb0ebfc2c43f3622046a, NAME => 'testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:55:35,539 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:35,540 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing ed051d3f7e34cb0ebfc2c43f3622046a, disabling compactions & flushes 2023-02-09 22:55:35,540 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:35,540 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:35,540 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. after waiting 0 ms 2023-02-09 22:55:35,540 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:35,540 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:35,540 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for ed051d3f7e34cb0ebfc2c43f3622046a: Waiting for close lock at 1675983335540Disabling compacts and flushes for region at 1675983335540Disabling writes for close at 1675983335540Writing region close event to WAL at 1675983335540Closed at 1675983335540 2023-02-09 22:55:35,542 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:35,543 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675983335543"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983335543"}]},"ts":"1675983335543"} 2023-02-09 22:55:35,544 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:55:35,545 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:35,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983335545"}]},"ts":"1675983335545"} 2023-02-09 22:55:35,547 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-09 22:55:35,551 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:55:35,552 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:55:35,552 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:55:35,552 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:55:35,552 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:55:35,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ed051d3f7e34cb0ebfc2c43f3622046a, ASSIGN}] 2023-02-09 22:55:35,554 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ed051d3f7e34cb0ebfc2c43f3622046a, ASSIGN 2023-02-09 22:55:35,555 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=ed051d3f7e34cb0ebfc2c43f3622046a, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,41885,1675983241428; forceNewPlan=false, retain=false 2023-02-09 22:55:35,705 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:55:35,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ed051d3f7e34cb0ebfc2c43f3622046a, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:35,706 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675983335706"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983335706"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983335706"}]},"ts":"1675983335706"} 2023-02-09 22:55:35,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure ed051d3f7e34cb0ebfc2c43f3622046a, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:55:35,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:35,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ed051d3f7e34cb0ebfc2c43f3622046a, NAME => 'testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:35,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:35,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:35,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,865 INFO [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,866 DEBUG [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1 2023-02-09 22:55:35,866 DEBUG [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1 2023-02-09 22:55:35,866 INFO [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed051d3f7e34cb0ebfc2c43f3622046a columnFamilyName cf1 2023-02-09 22:55:35,866 INFO [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] regionserver.HStore(310): Store=ed051d3f7e34cb0ebfc2c43f3622046a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:35,867 INFO [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,868 DEBUG [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2 2023-02-09 22:55:35,868 DEBUG [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2 2023-02-09 22:55:35,868 INFO [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed051d3f7e34cb0ebfc2c43f3622046a columnFamilyName cf2 2023-02-09 22:55:35,869 INFO [StoreOpener-ed051d3f7e34cb0ebfc2c43f3622046a-1] regionserver.HStore(310): Store=ed051d3f7e34cb0ebfc2c43f3622046a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:35,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:35,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:35,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:35,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened ed051d3f7e34cb0ebfc2c43f3622046a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61587798, jitterRate=-0.0822702944278717}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:35,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for ed051d3f7e34cb0ebfc2c43f3622046a: Running coprocessor pre-open hook at 1675983335864Writing region info on filesystem at 1675983335864Initializing all the Stores at 1675983335864Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983335864Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983335865 (+1 ms)Cleaning up temporary data from old regions at 1675983335870 (+5 ms)Cleaning up detritus from prior splits at 1675983335870Running coprocessor post-open hooks at 1675983335874 (+4 ms)Region opened successfully at 1675983335874 2023-02-09 22:55:35,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a., pid=104, masterSystemTime=1675983335859 2023-02-09 22:55:35,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:35,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:35,876 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ed051d3f7e34cb0ebfc2c43f3622046a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:35,877 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675983335876"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983335876"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983335876"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983335876"}]},"ts":"1675983335876"} 2023-02-09 22:55:35,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-09 22:55:35,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure ed051d3f7e34cb0ebfc2c43f3622046a, server=jenkins-hbase9.apache.org,41885,1675983241428 in 170 msec 2023-02-09 22:55:35,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-09 22:55:35,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ed051d3f7e34cb0ebfc2c43f3622046a, ASSIGN in 326 msec 2023-02-09 22:55:35,881 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:35,881 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983335881"}]},"ts":"1675983335881"} 2023-02-09 22:55:35,882 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-09 22:55:35,884 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:35,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 374 msec 2023-02-09 22:55:36,019 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:36,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 22:55:36,020 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 22:55:36,020 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,020 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 22:55:36,020 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 22:55:36,021 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,021 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,022 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,022 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,022 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,022 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,023 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,023 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,024 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,024 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,024 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,024 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,024 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,024 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:36,025 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:36,025 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,025 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. started... 2023-02-09 22:55:36,027 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ed051d3f7e34cb0ebfc2c43f3622046a 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:55:36,027 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,029 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,029 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,029 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,029 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,029 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,037 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/e8cb7da9407c49e98aebb89de03adf9a 2023-02-09 22:55:36,047 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/153841eaa7714310b2384754028f6769 2023-02-09 22:55:36,051 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/e8cb7da9407c49e98aebb89de03adf9a as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/e8cb7da9407c49e98aebb89de03adf9a 2023-02-09 22:55:36,054 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/e8cb7da9407c49e98aebb89de03adf9a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:36,055 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/153841eaa7714310b2384754028f6769 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/153841eaa7714310b2384754028f6769 2023-02-09 22:55:36,058 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/153841eaa7714310b2384754028f6769, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:36,059 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ed051d3f7e34cb0ebfc2c43f3622046a in 32ms, sequenceid=6, compaction requested=false 2023-02-09 22:55:36,059 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-09 22:55:36,060 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ed051d3f7e34cb0ebfc2c43f3622046a: 2023-02-09 22:55:36,060 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,060 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:36,060 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:36,060 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,060 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,060 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,061 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,061 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,062 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,062 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 22:55:36,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@371aedb3[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:36,064 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 22:55:36,064 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,065 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:36,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,065 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,066 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:36,068 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,068 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,068 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,069 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,069 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,069 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,069 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,069 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,069 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,069 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:55:36,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 22:55:36,072 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 22:55:36,072 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:36,072 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:36,072 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 22:55:36,072 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:36,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,073 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,074 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,074 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,074 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,074 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,075 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,076 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,076 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,077 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,079 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 22:55:36,079 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,079 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:36,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,079 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,080 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,180 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:36,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:36,191 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:36,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:36,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 22:55:36,193 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 22:55:36,193 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,193 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 22:55:36,193 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 22:55:36,193 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,193 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,195 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,195 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,195 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,195 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,196 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,196 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,197 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,197 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,197 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,197 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,197 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,197 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:36,198 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:36,198 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,198 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. started... 2023-02-09 22:55:36,198 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ed051d3f7e34cb0ebfc2c43f3622046a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:36,199 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,203 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,203 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,203 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,203 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,203 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,206 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9 2023-02-09 22:55:36,221 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/443a86a1386c48579947cd2917b702c3 2023-02-09 22:55:36,225 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9 2023-02-09 22:55:36,230 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:36,230 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/443a86a1386c48579947cd2917b702c3 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/443a86a1386c48579947cd2917b702c3 2023-02-09 22:55:36,234 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/443a86a1386c48579947cd2917b702c3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:36,235 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed051d3f7e34cb0ebfc2c43f3622046a in 37ms, sequenceid=19, compaction requested=false 2023-02-09 22:55:36,235 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ed051d3f7e34cb0ebfc2c43f3622046a: 2023-02-09 22:55:36,235 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,235 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:36,235 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:36,235 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,235 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,235 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,238 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,238 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,239 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,239 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 22:55:36,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a35a9c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:36,240 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 22:55:36,240 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:36,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:36,242 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:36,244 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,244 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,244 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,244 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,244 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,244 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,245 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,245 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,245 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,245 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:55:36,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 22:55:36,248 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 22:55:36,248 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:36,248 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:36,248 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 22:55:36,248 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:36,251 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,251 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,251 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,251 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,252 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,252 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,252 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,254 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,254 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,255 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,258 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 22:55:36,258 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:36,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:36,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,257 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,258 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,358 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:36,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:36,370 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:36,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:36,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 22:55:36,372 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 22:55:36,372 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,372 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 22:55:36,372 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 22:55:36,372 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,372 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,374 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,374 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,374 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,374 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,374 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,374 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:36,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,375 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,375 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,376 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,376 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,376 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. started... 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,376 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ed051d3f7e34cb0ebfc2c43f3622046a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,376 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,377 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,379 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,379 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,379 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,379 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,379 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,385 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/94a8f880d6c941efaf2bf99bd914b307 2023-02-09 22:55:36,395 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/3fdf6405dc6e4b6bb34443cef1adc3fa 2023-02-09 22:55:36,399 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/94a8f880d6c941efaf2bf99bd914b307 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/94a8f880d6c941efaf2bf99bd914b307 2023-02-09 22:55:36,403 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/94a8f880d6c941efaf2bf99bd914b307, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:36,403 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/3fdf6405dc6e4b6bb34443cef1adc3fa as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/3fdf6405dc6e4b6bb34443cef1adc3fa 2023-02-09 22:55:36,407 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/3fdf6405dc6e4b6bb34443cef1adc3fa, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:36,408 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed051d3f7e34cb0ebfc2c43f3622046a in 32ms, sequenceid=32, compaction requested=true 2023-02-09 22:55:36,408 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ed051d3f7e34cb0ebfc2c43f3622046a: 2023-02-09 22:55:36,408 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,408 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:36,408 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:36,408 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,408 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,408 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,410 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,410 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,410 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,410 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 22:55:36,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14eec9be[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:36,412 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 22:55:36,412 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,413 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:36,413 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,413 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,413 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,414 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,413 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,414 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:36,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,414 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:36,414 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,414 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,414 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:36,415 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,415 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,415 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,416 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,417 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,417 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,416 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,417 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,417 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,417 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:55:36,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 22:55:36,420 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 22:55:36,420 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:36,420 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:36,420 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 22:55:36,420 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:36,421 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,421 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,421 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,421 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,421 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,422 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,421 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,422 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,422 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,422 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,424 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,424 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,425 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,430 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,430 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 22:55:36,430 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:36,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,430 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:36,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,431 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,531 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:36,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:36,543 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:36,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:36,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 22:55:36,544 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 22:55:36,544 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,544 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 22:55:36,544 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 22:55:36,545 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,545 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,547 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,547 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,547 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,547 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,547 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,547 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:36,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:36,549 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:36,550 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,550 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. started... 2023-02-09 22:55:36,550 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ed051d3f7e34cb0ebfc2c43f3622046a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:36,551 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,552 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,552 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,552 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,552 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,552 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,558 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/1f99f9e74f2d4cf6ac96a92a8fd819f3 2023-02-09 22:55:36,568 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/84d21dfaec62477ba87e2dc40c363397 2023-02-09 22:55:36,571 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/1f99f9e74f2d4cf6ac96a92a8fd819f3 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/1f99f9e74f2d4cf6ac96a92a8fd819f3 2023-02-09 22:55:36,575 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/1f99f9e74f2d4cf6ac96a92a8fd819f3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:36,576 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/84d21dfaec62477ba87e2dc40c363397 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/84d21dfaec62477ba87e2dc40c363397 2023-02-09 22:55:36,580 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/84d21dfaec62477ba87e2dc40c363397, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:36,581 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed051d3f7e34cb0ebfc2c43f3622046a in 31ms, sequenceid=45, compaction requested=true 2023-02-09 22:55:36,581 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ed051d3f7e34cb0ebfc2c43f3622046a: 2023-02-09 22:55:36,581 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,581 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:36,581 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:36,581 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,581 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,581 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,583 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,583 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,583 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,583 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 22:55:36,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58c2a9b3[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:36,585 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 22:55:36,585 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:36,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,588 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,588 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:36,587 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,588 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:36,588 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:36,589 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,589 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,589 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,589 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,589 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,589 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,589 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,590 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,590 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,590 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:55:36,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 22:55:36,592 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 22:55:36,593 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:36,593 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:36,593 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 22:55:36,593 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:36,594 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,594 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,595 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,595 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,596 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,597 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,597 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,598 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,601 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,601 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 22:55:36,602 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,601 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,602 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:36,602 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,602 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,602 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,602 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:36,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,602 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,603 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,603 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,603 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,603 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,603 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,703 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:36,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:36,714 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:36,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:36,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 22:55:36,715 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 22:55:36,715 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,715 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 22:55:36,715 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 22:55:36,716 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,716 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,717 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,717 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,717 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,717 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,717 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,718 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,719 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,719 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,719 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,719 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,719 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,719 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:36,719 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:36,720 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,720 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. started... 2023-02-09 22:55:36,720 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ed051d3f7e34cb0ebfc2c43f3622046a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:36,721 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,721 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,721 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,721 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,721 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,721 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,728 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/c7295651f9054bd1a78593e38eec0f7e 2023-02-09 22:55:36,738 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/18fbee0fe3c14adf91cb45507917725a 2023-02-09 22:55:36,741 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/c7295651f9054bd1a78593e38eec0f7e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/c7295651f9054bd1a78593e38eec0f7e 2023-02-09 22:55:36,744 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/c7295651f9054bd1a78593e38eec0f7e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:36,745 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/18fbee0fe3c14adf91cb45507917725a as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/18fbee0fe3c14adf91cb45507917725a 2023-02-09 22:55:36,748 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/18fbee0fe3c14adf91cb45507917725a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:36,749 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed051d3f7e34cb0ebfc2c43f3622046a in 29ms, sequenceid=58, compaction requested=true 2023-02-09 22:55:36,749 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ed051d3f7e34cb0ebfc2c43f3622046a: 2023-02-09 22:55:36,749 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,749 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:36,749 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:36,749 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,749 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,749 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,751 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,751 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,751 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 22:55:36,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38d5a06d[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:36,753 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 22:55:36,753 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,755 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,755 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,755 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:36,755 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,755 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:36,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,755 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:36,755 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,756 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,756 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:36,761 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,761 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,761 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,761 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,761 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,761 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,761 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,765 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,765 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,765 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:55:36,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 22:55:36,768 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 22:55:36,768 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:36,768 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:36,768 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 22:55:36,768 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:36,769 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,769 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,769 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,769 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,769 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,769 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,770 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,770 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,770 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,771 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,772 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,773 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,773 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,775 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,775 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,775 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,775 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,776 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,776 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 22:55:36,776 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:36,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,776 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,776 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,775 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,775 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,776 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,776 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:36,776 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:36,776 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,776 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,777 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,777 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,777 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,777 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,777 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,777 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,876 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:36,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:36,889 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:36,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:36,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 22:55:36,890 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 22:55:36,890 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,890 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 22:55:36,890 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 22:55:36,890 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,890 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,892 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,892 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,892 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,892 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,892 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,893 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 22:55:36,893 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,893 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 22:55:36,893 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,893 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,893 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:36,894 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:36,894 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,894 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. started... 2023-02-09 22:55:36,894 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ed051d3f7e34cb0ebfc2c43f3622046a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:36,895 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,895 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,896 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,896 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,896 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,896 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,902 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/457a0b0c109f4ecca134c7970c8a6506 2023-02-09 22:55:36,912 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/788b48965d304f748c50cc1218c52afd 2023-02-09 22:55:36,915 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/457a0b0c109f4ecca134c7970c8a6506 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/457a0b0c109f4ecca134c7970c8a6506 2023-02-09 22:55:36,918 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/457a0b0c109f4ecca134c7970c8a6506, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:36,919 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/788b48965d304f748c50cc1218c52afd as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/788b48965d304f748c50cc1218c52afd 2023-02-09 22:55:36,922 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/788b48965d304f748c50cc1218c52afd, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:36,923 INFO [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed051d3f7e34cb0ebfc2c43f3622046a in 29ms, sequenceid=71, compaction requested=true 2023-02-09 22:55:36,923 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ed051d3f7e34cb0ebfc2c43f3622046a: 2023-02-09 22:55:36,923 DEBUG [rs(jenkins-hbase9.apache.org,41885,1675983241428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:36,923 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:36,923 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:36,923 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,923 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 22:55:36,923 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 22:55:36,925 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:36,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 22:55:36,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22386a51[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:36,927 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 22:55:36,927 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,928 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,928 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:36,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 22:55:36,928 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 22:55:36,929 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:36,933 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,933 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,933 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,934 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,934 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,934 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 22:55:36,934 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,934 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,934 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,934 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 22:55:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase9.apache.org,41885,1675983241428': 2023-02-09 22:55:36,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 22:55:36,937 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 22:55:36,937 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:36,937 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:36,937 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 22:55:36,937 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:36,939 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,939 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,939 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,939 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,939 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,940 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:36,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,940 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,941 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,942 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:36,942 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 22:55:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,944 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 22:55:36,944 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:36,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:36,944 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:36,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:36,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:36,944 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:36,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 22:55:36,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:36,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 22:55:36,945 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 22:55:37,045 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:37,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:37,052 INFO [Listener at localhost/44743] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:37,053 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.StoreUtils(129): cannot split hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9 because midkey is the same as first or last row 2023-02-09 22:55:37,053 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.StoreUtils(129): cannot split hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/443a86a1386c48579947cd2917b702c3 because midkey is the same as first or last row 2023-02-09 22:55:37,053 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.HRegion(2744): Flushing ed051d3f7e34cb0ebfc2c43f3622046a 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 22:55:37,062 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/7db66b9dd7894e1d8854f02860d93731 2023-02-09 22:55:37,072 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/6b38627cee364da59888c0d449cfcee1 2023-02-09 22:55:37,076 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf1/7db66b9dd7894e1d8854f02860d93731 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/7db66b9dd7894e1d8854f02860d93731 2023-02-09 22:55:37,078 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/7db66b9dd7894e1d8854f02860d93731, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:37,079 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/.tmp/cf2/6b38627cee364da59888c0d449cfcee1 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/6b38627cee364da59888c0d449cfcee1 2023-02-09 22:55:37,082 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/6b38627cee364da59888c0d449cfcee1, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:37,083 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ed051d3f7e34cb0ebfc2c43f3622046a in 30ms, sequenceid=82, compaction requested=true 2023-02-09 22:55:37,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.HRegion(2445): Flush status journal for ed051d3f7e34cb0ebfc2c43f3622046a: 2023-02-09 22:55:37,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.StoreUtils(129): cannot split hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9 because midkey is the same as first or last row 2023-02-09 22:55:37,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41885] regionserver.StoreUtils(129): cannot split hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/443a86a1386c48579947cd2917b702c3 because midkey is the same as first or last row 2023-02-09 22:55:37,084 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:37,084 DEBUG [Listener at localhost/44743] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-09 22:55:37,084 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:37,086 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-09 22:55:37,086 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testInvalidSplitKey 2023-02-09 22:55:37,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:37,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:37,089 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983337089"}]},"ts":"1675983337089"} 2023-02-09 22:55:37,090 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-09 22:55:37,092 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-09 22:55:37,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ed051d3f7e34cb0ebfc2c43f3622046a, UNASSIGN}] 2023-02-09 22:55:37,093 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ed051d3f7e34cb0ebfc2c43f3622046a, UNASSIGN 2023-02-09 22:55:37,094 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ed051d3f7e34cb0ebfc2c43f3622046a, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:37,094 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675983337094"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983337094"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983337094"}]},"ts":"1675983337094"} 2023-02-09 22:55:37,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure ed051d3f7e34cb0ebfc2c43f3622046a, server=jenkins-hbase9.apache.org,41885,1675983241428}] 2023-02-09 22:55:37,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:37,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing ed051d3f7e34cb0ebfc2c43f3622046a, disabling compactions & flushes 2023-02-09 22:55:37,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. after waiting 0 ms 2023-02-09 22:55:37,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:37,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 22:55:37,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:37,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a. 2023-02-09 22:55:37,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for ed051d3f7e34cb0ebfc2c43f3622046a: Waiting for close lock at 1675983337247Running coprocessor pre-close hooks at 1675983337247Disabling compacts and flushes for region at 1675983337247Disabling writes for close at 1675983337247Writing region close event to WAL at 1675983337248 (+1 ms)Running coprocessor post-close hooks at 1675983337251 (+3 ms)Closed at 1675983337251 2023-02-09 22:55:37,253 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:37,253 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ed051d3f7e34cb0ebfc2c43f3622046a, regionState=CLOSED 2023-02-09 22:55:37,253 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675983337253"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983337253"}]},"ts":"1675983337253"} 2023-02-09 22:55:37,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-09 22:55:37,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure ed051d3f7e34cb0ebfc2c43f3622046a, server=jenkins-hbase9.apache.org,41885,1675983241428 in 159 msec 2023-02-09 22:55:37,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-09 22:55:37,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ed051d3f7e34cb0ebfc2c43f3622046a, UNASSIGN in 163 msec 2023-02-09 22:55:37,258 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983337258"}]},"ts":"1675983337258"} 2023-02-09 22:55:37,259 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-09 22:55:37,261 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-09 22:55:37,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 176 msec 2023-02-09 22:55:37,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:37,391 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-09 22:55:37,392 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testInvalidSplitKey 2023-02-09 22:55:37,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:37,394 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:37,394 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:37,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 22:55:37,397 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:37,399 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/recovered.edits] 2023-02-09 22:55:37,401 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/05e270f1ee2e4d0aab58e7ae7790f4e9 2023-02-09 22:55:37,402 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/1f99f9e74f2d4cf6ac96a92a8fd819f3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/1f99f9e74f2d4cf6ac96a92a8fd819f3 2023-02-09 22:55:37,403 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/457a0b0c109f4ecca134c7970c8a6506 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/457a0b0c109f4ecca134c7970c8a6506 2023-02-09 22:55:37,404 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/7db66b9dd7894e1d8854f02860d93731 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/7db66b9dd7894e1d8854f02860d93731 2023-02-09 22:55:37,405 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/94a8f880d6c941efaf2bf99bd914b307 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/94a8f880d6c941efaf2bf99bd914b307 2023-02-09 22:55:37,406 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/c7295651f9054bd1a78593e38eec0f7e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/c7295651f9054bd1a78593e38eec0f7e 2023-02-09 22:55:37,407 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/e8cb7da9407c49e98aebb89de03adf9a to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf1/e8cb7da9407c49e98aebb89de03adf9a 2023-02-09 22:55:37,409 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/153841eaa7714310b2384754028f6769 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/153841eaa7714310b2384754028f6769 2023-02-09 22:55:37,410 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/18fbee0fe3c14adf91cb45507917725a to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/18fbee0fe3c14adf91cb45507917725a 2023-02-09 22:55:37,411 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/3fdf6405dc6e4b6bb34443cef1adc3fa to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/3fdf6405dc6e4b6bb34443cef1adc3fa 2023-02-09 22:55:37,412 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/443a86a1386c48579947cd2917b702c3 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/443a86a1386c48579947cd2917b702c3 2023-02-09 22:55:37,413 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/6b38627cee364da59888c0d449cfcee1 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/6b38627cee364da59888c0d449cfcee1 2023-02-09 22:55:37,414 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/788b48965d304f748c50cc1218c52afd to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/788b48965d304f748c50cc1218c52afd 2023-02-09 22:55:37,415 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/84d21dfaec62477ba87e2dc40c363397 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/cf2/84d21dfaec62477ba87e2dc40c363397 2023-02-09 22:55:37,417 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/recovered.edits/85.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a/recovered.edits/85.seqid 2023-02-09 22:55:37,417 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testInvalidSplitKey/ed051d3f7e34cb0ebfc2c43f3622046a 2023-02-09 22:55:37,417 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 22:55:37,419 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:37,420 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-09 22:55:37,424 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-09 22:55:37,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:37,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-09 22:55:37,425 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983337425"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:37,426 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 22:55:37,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ed051d3f7e34cb0ebfc2c43f3622046a, NAME => 'testInvalidSplitKey,,1675983335510.ed051d3f7e34cb0ebfc2c43f3622046a.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 22:55:37,426 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-09 22:55:37,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983337426"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:37,427 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-09 22:55:37,429 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:37,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 37 msec 2023-02-09 22:55:37,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 22:55:37,496 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-09 22:55:37,509 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=368 (was 346) - Thread LEAK? -, OpenFileDescriptor=745 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 104), ProcessCount=168 (was 168), AvailableMemoryMB=10676 (was 10683) 2023-02-09 22:55:37,521 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=368, OpenFileDescriptor=745, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=168, AvailableMemoryMB=10676 2023-02-09 22:55:37,521 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:37,524 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:55:37,524 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:55:37,525 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:37,527 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:37,527 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:37,528 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,529 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541 empty. 2023-02-09 22:55:37,529 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,529 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 22:55:37,537 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:37,538 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 98929a422906bc0bcc11a422f24fe541, NAME => 'testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:55:37,545 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:37,545 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 98929a422906bc0bcc11a422f24fe541, disabling compactions & flushes 2023-02-09 22:55:37,545 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:37,545 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:37,545 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. after waiting 0 ms 2023-02-09 22:55:37,545 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:37,545 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:37,545 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 98929a422906bc0bcc11a422f24fe541: Waiting for close lock at 1675983337545Disabling compacts and flushes for region at 1675983337545Disabling writes for close at 1675983337545Writing region close event to WAL at 1675983337545Closed at 1675983337545 2023-02-09 22:55:37,547 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:37,547 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983337547"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983337547"}]},"ts":"1675983337547"} 2023-02-09 22:55:37,548 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:55:37,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:37,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983337549"}]},"ts":"1675983337549"} 2023-02-09 22:55:37,550 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 22:55:37,553 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:55:37,554 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:55:37,554 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:55:37,554 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:55:37,554 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:55:37,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, ASSIGN}] 2023-02-09 22:55:37,556 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, ASSIGN 2023-02-09 22:55:37,556 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:55:37,706 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:55:37,708 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=98929a422906bc0bcc11a422f24fe541, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:37,708 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983337707"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983337707"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983337707"}]},"ts":"1675983337707"} 2023-02-09 22:55:37,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:37,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:37,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 98929a422906bc0bcc11a422f24fe541, NAME => 'testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:37,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:37,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:37,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,866 INFO [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,867 DEBUG [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1 2023-02-09 22:55:37,867 DEBUG [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1 2023-02-09 22:55:37,867 INFO [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 98929a422906bc0bcc11a422f24fe541 columnFamilyName cf1 2023-02-09 22:55:37,868 INFO [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] regionserver.HStore(310): Store=98929a422906bc0bcc11a422f24fe541/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:37,868 INFO [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,869 DEBUG [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2 2023-02-09 22:55:37,869 DEBUG [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2 2023-02-09 22:55:37,869 INFO [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 98929a422906bc0bcc11a422f24fe541 columnFamilyName cf2 2023-02-09 22:55:37,869 INFO [StoreOpener-98929a422906bc0bcc11a422f24fe541-1] regionserver.HStore(310): Store=98929a422906bc0bcc11a422f24fe541/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:37,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:37,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:37,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:37,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 98929a422906bc0bcc11a422f24fe541; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63457356, jitterRate=-0.05441170930862427}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:37,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 98929a422906bc0bcc11a422f24fe541: Running coprocessor pre-open hook at 1675983337865Writing region info on filesystem at 1675983337865Initializing all the Stores at 1675983337865Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983337865Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983337865Cleaning up temporary data from old regions at 1675983337871 (+6 ms)Cleaning up detritus from prior splits at 1675983337871Running coprocessor post-open hooks at 1675983337874 (+3 ms)Region opened successfully at 1675983337874 2023-02-09 22:55:37,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541., pid=111, masterSystemTime=1675983337860 2023-02-09 22:55:37,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:37,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:37,877 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=98929a422906bc0bcc11a422f24fe541, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:37,877 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983337877"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983337877"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983337877"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983337877"}]},"ts":"1675983337877"} 2023-02-09 22:55:37,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-09 22:55:37,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 in 169 msec 2023-02-09 22:55:37,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-09 22:55:37,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, ASSIGN in 325 msec 2023-02-09 22:55:37,881 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:37,881 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983337881"}]},"ts":"1675983337881"} 2023-02-09 22:55:37,882 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 22:55:37,884 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:37,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 359 msec 2023-02-09 22:55:38,033 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:38,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 22:55:38,034 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,034 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,034 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 22:55:38,034 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:55:38,035 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,035 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,036 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,036 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,036 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,036 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,037 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,037 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,037 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,038 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,038 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,038 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,038 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,038 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,038 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,038 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,040 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,041 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:38,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:38,042 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,042 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. started... 2023-02-09 22:55:38,043 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98929a422906bc0bcc11a422f24fe541 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:55:38,043 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,044 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,044 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,053 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/ef378bcd97e641fc8d71a701fe9eec4c 2023-02-09 22:55:38,063 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/c674402dad904a228c16b34bc1d36958 2023-02-09 22:55:38,067 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/ef378bcd97e641fc8d71a701fe9eec4c as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c 2023-02-09 22:55:38,070 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:38,070 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/c674402dad904a228c16b34bc1d36958 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958 2023-02-09 22:55:38,073 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:38,075 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 32ms, sequenceid=6, compaction requested=false 2023-02-09 22:55:38,075 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,076 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98929a422906bc0bcc11a422f24fe541: 2023-02-09 22:55:38,076 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,076 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:38,076 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:38,076 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,076 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,076 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,078 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,078 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,079 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,079 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 22:55:38,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40f9e491[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:38,081 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:55:38,081 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,083 ERROR [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid 73c0fc0c-d116-48f6-bf3b-b31855654d02) service to localhost/127.0.0.1:41659 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:55:38,084 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,083 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:38,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,084 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:38,086 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,086 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,087 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,087 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,087 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,087 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,087 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,088 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,088 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,088 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:38,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:55:38,090 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 22:55:38,090 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:38,090 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:38,090 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,091 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:38,092 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,092 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,092 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,092 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,092 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,092 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,093 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,092 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,093 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,093 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,095 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,095 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,095 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,099 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,099 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,098 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,099 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,099 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,099 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:38,099 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,099 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:38,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,099 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,099 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,100 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,100 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,100 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,100 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,199 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:38,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:38,210 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:38,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:38,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 22:55:38,212 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,212 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,212 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 22:55:38,212 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:55:38,212 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,212 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,215 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,215 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,215 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,215 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,215 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,215 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,216 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:38,217 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:38,217 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,217 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. started... 2023-02-09 22:55:38,218 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98929a422906bc0bcc11a422f24fe541 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:38,219 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,219 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,219 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,219 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,219 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,219 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,231 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/d6f1c50f1504489b8a0bcf85c56c836c 2023-02-09 22:55:38,249 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/3f8a5db51e7a4a12bd090effb82a931c 2023-02-09 22:55:38,254 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/d6f1c50f1504489b8a0bcf85c56c836c as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c 2023-02-09 22:55:38,258 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:38,258 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/3f8a5db51e7a4a12bd090effb82a931c as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c 2023-02-09 22:55:38,262 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:38,265 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 47ms, sequenceid=19, compaction requested=false 2023-02-09 22:55:38,265 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98929a422906bc0bcc11a422f24fe541: 2023-02-09 22:55:38,265 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,265 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:38,265 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:38,266 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,266 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,266 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,270 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,270 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,271 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,271 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 22:55:38,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d3b420c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:38,273 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:55:38,273 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:38,276 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:38,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,276 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,277 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:38,278 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,278 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,278 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,279 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,281 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,282 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,282 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,282 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:38,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:55:38,287 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 22:55:38,287 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:38,287 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:38,287 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,287 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:38,289 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,289 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,290 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,290 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,291 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,293 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,294 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,294 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,298 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,298 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,298 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,298 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,299 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:38,299 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,299 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:38,299 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,399 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:38,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:38,412 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:38,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:38,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 22:55:38,413 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,413 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,413 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 22:55:38,413 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:55:38,414 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,414 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,415 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,415 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,415 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,415 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,416 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,416 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,417 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,417 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,418 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:38,418 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:38,418 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,418 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. started... 2023-02-09 22:55:38,418 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98929a422906bc0bcc11a422f24fe541 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:38,420 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,422 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,422 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,422 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,422 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,422 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,428 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/f4eaa7094af744cf829be1bcedcb0a8b 2023-02-09 22:55:38,447 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/4fd3d2ca44cc4d239bb0f6780b050286 2023-02-09 22:55:38,452 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/f4eaa7094af744cf829be1bcedcb0a8b as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b 2023-02-09 22:55:38,455 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:38,456 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/4fd3d2ca44cc4d239bb0f6780b050286 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286 2023-02-09 22:55:38,460 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:38,461 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 43ms, sequenceid=32, compaction requested=true 2023-02-09 22:55:38,461 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98929a422906bc0bcc11a422f24fe541: 2023-02-09 22:55:38,461 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,461 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:38,461 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:38,461 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,461 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,461 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,464 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,464 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,465 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,465 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 22:55:38,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18784d0e[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:38,467 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:55:38,468 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,469 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,469 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,469 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:38,469 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,469 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:38,470 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,469 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,470 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:38,470 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,470 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:38,472 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,472 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,472 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,473 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,472 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,473 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,473 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,472 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,473 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,473 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:38,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:55:38,477 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 22:55:38,477 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:38,477 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:38,477 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,477 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:38,480 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,480 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,480 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,481 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,482 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,484 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,485 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,485 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,491 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,491 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,491 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,492 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:38,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,491 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,492 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:38,492 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,492 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,492 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,492 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,493 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,493 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,493 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,493 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,493 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,592 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:38,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:38,613 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:38,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:38,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 22:55:38,614 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,614 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,614 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 22:55:38,614 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:55:38,615 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,615 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,616 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,616 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,616 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,616 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,617 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,617 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,618 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,618 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,619 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,619 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,619 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,619 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. started... 2023-02-09 22:55:38,619 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98929a422906bc0bcc11a422f24fe541 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:38,623 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,623 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,623 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,632 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/899d6c7c2db04e51867e1ad6ae79e565 2023-02-09 22:55:38,656 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/cd606858ec664914b5c555b9bf13d9cb 2023-02-09 22:55:38,661 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/899d6c7c2db04e51867e1ad6ae79e565 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565 2023-02-09 22:55:38,665 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:38,666 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/cd606858ec664914b5c555b9bf13d9cb as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb 2023-02-09 22:55:38,669 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:38,672 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 53ms, sequenceid=45, compaction requested=true 2023-02-09 22:55:38,672 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98929a422906bc0bcc11a422f24fe541: 2023-02-09 22:55:38,672 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,672 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:38,672 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:38,672 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,672 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,672 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,674 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,674 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,675 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,675 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 22:55:38,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7973c6b2[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:38,676 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:55:38,676 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,679 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:38,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,679 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,679 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,679 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,679 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,679 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:38,679 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:38,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,680 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,680 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,681 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:38,681 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,681 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,681 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,682 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,682 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,682 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,683 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,683 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,683 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,683 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:38,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:55:38,687 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 22:55:38,687 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:38,687 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:38,687 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,687 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:38,694 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,694 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,694 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,695 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,695 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,699 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,701 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,701 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,701 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,704 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,704 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,704 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,704 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,704 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,705 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,705 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,705 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:38,705 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:38,706 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:38,711 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,712 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution because node does not exist (not an error) 2023-02-09 22:55:38,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,806 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:38,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:38,818 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:38,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:38,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 22:55:38,819 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,820 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,820 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 22:55:38,820 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:55:38,820 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,820 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,822 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,822 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,822 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,823 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,823 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,823 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,823 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,823 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:38,824 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:38,824 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,824 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. started... 2023-02-09 22:55:38,824 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98929a422906bc0bcc11a422f24fe541 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:38,825 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,830 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,830 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,831 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,831 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,831 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,836 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa 2023-02-09 22:55:38,847 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/97e42b893c994c21896bc4210054edcc 2023-02-09 22:55:38,851 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa 2023-02-09 22:55:38,855 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:38,856 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/97e42b893c994c21896bc4210054edcc as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc 2023-02-09 22:55:38,859 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:38,860 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 36ms, sequenceid=58, compaction requested=true 2023-02-09 22:55:38,860 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98929a422906bc0bcc11a422f24fe541: 2023-02-09 22:55:38,860 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:38,860 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:38,860 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:38,860 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,860 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:38,861 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:38,865 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,865 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 22:55:38,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76a17861[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:38,867 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:55:38,867 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,869 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:38,869 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:38,873 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,873 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,873 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,873 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,873 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,873 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,873 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,874 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:38,874 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,874 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:38,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:55:38,877 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 22:55:38,877 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:38,877 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:38,877 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 22:55:38,877 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:38,878 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,878 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,878 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,878 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,878 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,878 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,879 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,879 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,879 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:38,880 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:38,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,881 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,881 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:38,882 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:38,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:38,885 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,885 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:38,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:38,885 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:38,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:38,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,885 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:38,886 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:38,886 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:38,886 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,886 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,986 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:38,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:38,997 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:38,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:38,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 22:55:38,999 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:38,999 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:38,999 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 22:55:38,999 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 22:55:38,999 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:38,999 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,001 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:39,001 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:39,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:39,001 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:39,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:39,001 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:39,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:39,002 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,002 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,002 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:39,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 22:55:39,003 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:39,003 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:39,003 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:39,003 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 22:55:39,003 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:39,003 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:39,004 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:39,004 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:39,004 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. started... 2023-02-09 22:55:39,004 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98929a422906bc0bcc11a422f24fe541 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:39,005 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,005 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,006 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,006 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,006 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:39,006 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:39,015 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/f0e1437a3cdb4a5397730806670697c8 2023-02-09 22:55:39,026 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/db062cdb5c8044eca8ebf5c914954a1e 2023-02-09 22:55:39,029 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/f0e1437a3cdb4a5397730806670697c8 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8 2023-02-09 22:55:39,032 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:39,033 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/db062cdb5c8044eca8ebf5c914954a1e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e 2023-02-09 22:55:39,036 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:39,036 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 32ms, sequenceid=71, compaction requested=true 2023-02-09 22:55:39,037 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98929a422906bc0bcc11a422f24fe541: 2023-02-09 22:55:39,037 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:39,037 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:39,037 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:39,037 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,037 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 22:55:39,037 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 22:55:39,038 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,038 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:39,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:39,039 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,039 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:39,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:39,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:39,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:39,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:39,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 22:55:39,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21dd7f9b[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:39,041 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 22:55:39,041 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:39,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:39,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:39,042 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,043 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:39,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,043 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:39,043 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 22:55:39,043 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:39,043 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 22:55:39,043 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:39,043 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:39,047 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:39,047 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,047 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:39,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:39,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:39,048 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:39,049 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:39,049 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:39,049 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:39,050 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 22:55:39,050 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:39,050 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 22:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:39,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 22:55:39,053 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 22:55:39,053 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:39,053 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:39,053 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 22:55:39,053 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:39,054 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,054 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:39,054 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:39,054 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:39,055 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:39,055 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:39,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:39,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:39,056 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:39,057 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 22:55:39,057 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,058 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,063 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:39,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:39,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:39,063 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:39,063 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,064 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,063 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,064 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 22:55:39,064 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:39,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:39,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:39,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:39,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:39,065 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:39,064 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:39,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:39,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:39,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:39,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:39,065 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:39,065 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 22:55:39,165 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:39,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:39,425 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 22:55:39,426 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:39,427 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,428 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:39,428 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:39,429 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:39,677 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,677 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:39,677 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:39,677 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:39,677 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-09 22:55:39,677 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:39,678 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:39,678 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:39,678 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:39,678 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:39,680 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 22:55:39,680 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:39,680 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:39,681 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:39,681 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:39,681 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:39,681 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:39,682 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:39,682 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:39,682 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:39,682 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:39,682 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:39,682 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:39,683 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:39,683 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:39,683 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:39,683 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:39,683 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:39,683 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:39,684 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:39,684 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:39,684 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:39,684 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:39,684 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:39,684 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:39,685 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:39,685 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,685 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,685 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:39,686 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,686 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,686 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:39,686 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:39,686 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:39,686 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:39,687 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,687 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:39,687 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,687 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,687 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,687 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-09 22:55:39,688 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:39,688 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:39,689 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983369689 2023-02-09 22:55:39,690 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:39,690 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,693 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:39,693 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:39,693 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:39,693 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983399693 2023-02-09 22:55:39,693 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983459693 2023-02-09 22:55:39,693 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:55:39,693 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:39,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:39,695 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:39,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:39,943 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,944 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:39,944 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:39,944 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:39,944 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:39,944 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:39,944 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:39,944 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:39,944 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:39,945 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:39,946 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 22:55:39,946 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:39,947 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:39,947 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:39,947 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:39,947 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:39,947 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:39,947 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:39,947 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:39,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:39,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:39,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:39,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:39,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:39,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:39,948 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:39,949 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:39,950 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:39,950 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:39,950 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,950 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,950 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:39,950 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,950 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:39,951 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:39,951 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:39,951 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:39,951 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:39,951 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,951 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:39,951 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,951 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,951 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,951 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:55:39,952 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:39,952 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:39,953 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983369953 2023-02-09 22:55:39,954 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,954 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:39,956 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:39,957 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:39,957 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:39,957 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983399957 2023-02-09 22:55:39,957 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983459957 2023-02-09 22:55:39,957 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 22:55:39,957 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:39,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN}] 2023-02-09 22:55:39,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:39,959 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:39,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:39,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:40,207 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,207 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:40,207 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:40,207 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:40,207 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:40,208 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:40,208 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:40,208 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:40,208 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:40,208 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:40,209 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 22:55:40,210 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:40,210 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:40,210 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:40,210 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:40,211 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:40,211 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:40,211 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:40,211 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:40,211 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:40,211 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:40,211 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:40,212 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:40,212 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:40,212 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:40,212 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:40,212 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:40,212 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:40,212 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:40,213 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,214 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,214 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:40,214 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,214 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,214 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:40,214 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:40,214 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:40,215 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:40,215 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,215 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:40,215 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,215 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,215 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,215 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:55:40,216 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:40,216 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:40,217 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983370217 2023-02-09 22:55:40,217 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:40,218 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,220 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:40,220 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:40,220 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:40,221 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983400220 2023-02-09 22:55:40,221 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983460221 2023-02-09 22:55:40,221 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:55:40,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN}] 2023-02-09 22:55:40,221 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:40,223 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:40,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=98929a422906bc0bcc11a422f24fe541, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:40,224 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983340223"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983340223"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983340223"}]},"ts":"1675983340223"} 2023-02-09 22:55:40,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:40,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 98929a422906bc0bcc11a422f24fe541, disabling compactions & flushes 2023-02-09 22:55:40,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:40,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:40,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. after waiting 0 ms 2023-02-09 22:55:40,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:40,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing 98929a422906bc0bcc11a422f24fe541 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 22:55:40,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/2ca6670211d343e8be0ecde51c86375f 2023-02-09 22:55:40,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/a0bcce43db68456f900fc90ad62b2f58 2023-02-09 22:55:40,404 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf1/2ca6670211d343e8be0ecde51c86375f as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f 2023-02-09 22:55:40,408 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:40,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/.tmp/cf2/a0bcce43db68456f900fc90ad62b2f58 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58 2023-02-09 22:55:40,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:40,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 38ms, sequenceid=82, compaction requested=true 2023-02-09 22:55:40,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 22:55:40,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:40,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. 2023-02-09 22:55:40,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 98929a422906bc0bcc11a422f24fe541: Waiting for close lock at 1675983340376Running coprocessor pre-close hooks at 1675983340376Disabling compacts and flushes for region at 1675983340376Disabling writes for close at 1675983340376Obtaining lock to block concurrent updates at 1675983340377 (+1 ms)Preparing flush snapshotting stores in 98929a422906bc0bcc11a422f24fe541 at 1675983340377Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675983340377Flushing stores of testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. at 1675983340377Flushing 98929a422906bc0bcc11a422f24fe541/cf1: creating writer at 1675983340378 (+1 ms)Flushing 98929a422906bc0bcc11a422f24fe541/cf1: appending metadata at 1675983340380 (+2 ms)Flushing 98929a422906bc0bcc11a422f24fe541/cf1: closing flushed file at 1675983340380Flushing 98929a422906bc0bcc11a422f24fe541/cf2: creating writer at 1675983340393 (+13 ms)Flushing 98929a422906bc0bcc11a422f24fe541/cf2: appending metadata at 1675983340395 (+2 ms)Flushing 98929a422906bc0bcc11a422f24fe541/cf2: closing flushed file at 1675983340395Flushing 98929a422906bc0bcc11a422f24fe541/cf1: reopening flushed file at 1675983340404 (+9 ms)Flushing 98929a422906bc0bcc11a422f24fe541/cf2: reopening flushed file at 1675983340409 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 98929a422906bc0bcc11a422f24fe541 in 38ms, sequenceid=82, compaction requested=true at 1675983340414 (+5 ms)Writing region close event to WAL at 1675983340415 (+1 ms)Running coprocessor post-close hooks at 1675983340418 (+3 ms)Closed at 1675983340419 (+1 ms) 2023-02-09 22:55:40,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,421 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED 2023-02-09 22:55:40,422 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983340421"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983340421"}]},"ts":"1675983340421"} 2023-02-09 22:55:40,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-09 22:55:40,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 in 198 msec 2023-02-09 22:55:40,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-09 22:55:40,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN in 203 msec 2023-02-09 22:55:40,428 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:40,428 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:40,428 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:40,471 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,471 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:40,471 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:40,471 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:40,471 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:40,471 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:40,471 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:40,471 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:40,472 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:40,472 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:40,473 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 22:55:40,473 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:40,473 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:40,474 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:40,474 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:40,474 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:40,474 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:40,474 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:40,475 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:40,475 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:40,475 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:40,475 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:40,475 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:40,475 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:40,476 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:40,476 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:40,476 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:40,476 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:40,476 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:40,477 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:40,477 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:40,477 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:40,477 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:40,477 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:40,478 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:40,478 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:40,478 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:40,478 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:40,479 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,479 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,479 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:40,480 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,480 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:40,480 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:40,480 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:40,480 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:40,481 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:40,481 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,481 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:40,481 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:40,481 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:40,481 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,481 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,481 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,481 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:40,481 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:40,481 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-09 22:55:40,482 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:40,482 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:40,483 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983370483 2023-02-09 22:55:40,483 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:40,485 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:40,487 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:40,487 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:40,487 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:40,487 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983400487 2023-02-09 22:55:40,487 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983460487 2023-02-09 22:55:40,487 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:55:40,490 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:40,498 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=98929a422906bc0bcc11a422f24fe541, threads=14 2023-02-09 22:55:40,498 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,498 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,498 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,498 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,498 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,498 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,498 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,499 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,499 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,499 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,499 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,499 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,499 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,500 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,623 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,626 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,635 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,635 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,639 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,639 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,647 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,651 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,659 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,663 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,663 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,663 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,667 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:40,801 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:55:41,027 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,027 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 98929a422906bc0bcc11a422f24fe541 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:55:41,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:41,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:41,056 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:41,238 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:41,238 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:41,239 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:41,240 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 22:55:41,240 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:41,240 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:41,240 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:41,241 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:41,241 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:41,241 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:41,241 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,241 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:41,241 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,241 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:41,242 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:41,243 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,244 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:41,245 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,245 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,245 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:41,245 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:41,246 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:55:41,246 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:41,247 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:41,247 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983371247 2023-02-09 22:55:41,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,248 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:41,251 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:41,251 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:41,251 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:41,251 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983401251 2023-02-09 22:55:41,251 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983461251 2023-02-09 22:55:41,251 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 22:55:41,259 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=98929a422906bc0bcc11a422f24fe541, threads=14 2023-02-09 22:55:41,259 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,260 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,261 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,261 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,261 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,261 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,261 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,362 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,364 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,365 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,372 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,372 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,378 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,383 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,386 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,387 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,387 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,390 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565 for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,390 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,390 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,390 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb for region: 98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:41,391 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 98929a422906bc0bcc11a422f24fe541 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:55:41,418 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:41,422 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:55:41,423 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:55:41,423 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:41,423 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:41,423 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:41,501 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,502 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:41,502 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:41,502 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:41,502 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:41,502 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:41,502 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:41,502 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:41,502 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:41,503 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:41,504 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 22:55:41,504 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:41,504 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:41,504 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:41,505 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:41,505 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:41,505 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:41,505 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,506 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:41,506 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,506 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,506 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,507 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:41,507 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,507 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,507 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,508 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:41,508 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:41,508 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,508 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,508 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,509 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:41,509 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,509 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,509 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,509 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:41,510 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,510 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,510 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,510 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,511 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:41,511 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,511 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,511 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,511 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,511 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,512 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:41,512 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,512 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:41,512 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:41,512 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:41,513 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,513 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,513 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,513 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:41,513 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:41,513 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-09 22:55:41,514 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:41,514 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:41,515 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983371515 2023-02-09 22:55:41,516 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,516 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:41,518 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:41,518 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:41,519 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:41,519 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983401519 2023-02-09 22:55:41,519 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983461519 2023-02-09 22:55:41,519 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 22:55:41,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:41,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:41,523 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:41,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:41,769 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,769 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:41,769 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:41,769 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:41,770 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:41,770 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:41,770 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:41,770 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:41,770 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:41,770 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:41,771 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 22:55:41,772 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:41,772 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:41,772 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:41,772 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:41,772 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:41,772 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:41,772 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,773 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,774 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:41,775 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,775 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:41,775 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,775 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,775 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:41,775 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,775 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:41,776 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:41,776 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,777 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,777 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,777 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:41,777 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:41,777 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:55:41,777 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:41,778 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:41,778 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983371778 2023-02-09 22:55:41,778 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:41,780 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,782 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:41,782 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:41,782 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:41,782 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983401782 2023-02-09 22:55:41,782 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983461782 2023-02-09 22:55:41,782 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:55:41,782 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:41,784 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983341784"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675983341784"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675983341784"}]},"ts":"1675983341784"} 2023-02-09 22:55:41,784 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983341784"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983341784"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983341784"}]},"ts":"1675983341784"} 2023-02-09 22:55:41,785 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983341784"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983341784"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983341784"}]},"ts":"1675983341784"} 2023-02-09 22:55:41,786 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:41,786 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:41,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:41,786 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:42,032 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,033 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:42,033 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:42,033 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:42,033 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:42,033 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:42,033 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:42,033 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:42,033 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:42,034 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:42,035 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 22:55:42,035 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:42,035 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:42,035 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:42,036 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:42,036 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:42,036 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:42,036 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,036 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:42,036 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,036 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,037 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,037 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:42,037 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,037 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,037 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,037 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,038 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,039 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:42,039 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,039 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,039 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,040 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:42,041 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,041 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:42,041 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:42,041 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:42,041 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,041 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,041 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,041 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:42,041 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:42,041 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 22:55:42,042 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:42,042 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:42,043 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983372043 2023-02-09 22:55:42,044 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,044 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:42,046 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:42,047 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:42,047 INFO [Listener at localhost/44743] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase9.apache.org,40413,1675983241505, table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:42,047 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=11c11386996225c96ffa8a1e7e902af2, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 22:55:42,047 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=0e00bf05e3c10e81903e3afc1d91fc88, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 22:55:42,047 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:42,047 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983402047 2023-02-09 22:55:42,047 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983462047 2023-02-09 22:55:42,047 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:55:42,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983342047"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675983342047"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675983342047"}]},"ts":"1675983342047"} 2023-02-09 22:55:42,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983342047"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983342047"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983342047"}]},"ts":"1675983342047"} 2023-02-09 22:55:42,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983342047"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983342047"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983342047"}]},"ts":"1675983342047"} 2023-02-09 22:55:42,050 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:42,051 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:42,051 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:42,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:42,297 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,298 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:42,298 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:42,298 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:42,298 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:42,298 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:42,298 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:42,298 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:42,298 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:42,299 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:42,300 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-09 22:55:42,300 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:42,300 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:42,300 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:42,301 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:42,301 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:42,301 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:42,301 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,301 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:42,301 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,301 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:42,302 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,303 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,304 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,305 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:42,305 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,305 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:42,305 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:42,305 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:42,305 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,305 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,305 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,305 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:42,305 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:42,305 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:55:42,306 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:42,306 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:42,307 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983372307 2023-02-09 22:55:42,307 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:42,308 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,310 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:42,310 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:42,311 INFO [Listener at localhost/44743] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase9.apache.org,40413,1675983241505, table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:42,311 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=11c11386996225c96ffa8a1e7e902af2, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 22:55:42,311 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=0e00bf05e3c10e81903e3afc1d91fc88, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 22:55:42,311 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:42,311 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983402311 2023-02-09 22:55:42,311 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983462311 2023-02-09 22:55:42,311 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:55:42,311 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:42,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, ASSIGN}] 2023-02-09 22:55:42,313 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:42,313 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:42,313 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:42,561 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,561 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:42,561 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:42,561 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:42,562 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:42,562 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:42,562 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:42,562 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:42,562 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:42,563 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:42,564 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-09 22:55:42,564 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:42,564 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:42,564 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:42,564 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:42,564 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:42,565 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,566 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,567 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,568 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:42,569 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:42,569 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:42,569 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,569 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,569 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,569 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:42,569 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:42,569 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 22:55:42,570 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:42,570 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:42,570 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983372570 2023-02-09 22:55:42,571 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:42,572 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,574 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:42,574 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:42,574 INFO [Listener at localhost/44743] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase9.apache.org,40413,1675983241505, table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:42,574 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=11c11386996225c96ffa8a1e7e902af2, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 22:55:42,574 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=0e00bf05e3c10e81903e3afc1d91fc88, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 22:55:42,574 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:42,575 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983402575 2023-02-09 22:55:42,575 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983462575 2023-02-09 22:55:42,575 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 22:55:42,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, ASSIGN}] 2023-02-09 22:55:42,575 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:42,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, ASSIGN 2023-02-09 22:55:42,577 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, ASSIGN; state=CLOSED, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:55:42,578 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, ASSIGN 2023-02-09 22:55:42,578 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, ASSIGN; state=CLOSED, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:55:42,728 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 22:55:42,729 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0e00bf05e3c10e81903e3afc1d91fc88, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:42,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983342729"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983342729"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983342729"}]},"ts":"1675983342729"} 2023-02-09 22:55:42,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 0e00bf05e3c10e81903e3afc1d91fc88, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:42,733 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=11c11386996225c96ffa8a1e7e902af2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:42,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983342732"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983342732"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983342732"}]},"ts":"1675983342732"} 2023-02-09 22:55:42,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 11c11386996225c96ffa8a1e7e902af2, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:42,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:42,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0e00bf05e3c10e81903e3afc1d91fc88, NAME => 'testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 22:55:42,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:42,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:42,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,886 INFO [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,887 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1 2023-02-09 22:55:42,887 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1 2023-02-09 22:55:42,887 INFO [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e00bf05e3c10e81903e3afc1d91fc88 columnFamilyName cf1 2023-02-09 22:55:42,899 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f-top 2023-02-09 22:55:42,902 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa-top 2023-02-09 22:55:42,905 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565-top 2023-02-09 22:55:42,907 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c-top 2023-02-09 22:55:42,910 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c-top 2023-02-09 22:55:42,913 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8-top 2023-02-09 22:55:42,916 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b-top 2023-02-09 22:55:42,916 INFO [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(310): Store=0e00bf05e3c10e81903e3afc1d91fc88/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:42,916 INFO [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,917 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2 2023-02-09 22:55:42,917 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2 2023-02-09 22:55:42,917 INFO [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e00bf05e3c10e81903e3afc1d91fc88 columnFamilyName cf2 2023-02-09 22:55:42,928 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c-top 2023-02-09 22:55:42,931 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286-top 2023-02-09 22:55:42,933 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc-top 2023-02-09 22:55:42,936 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58-top 2023-02-09 22:55:42,939 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958-top 2023-02-09 22:55:42,942 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb-top 2023-02-09 22:55:42,945 DEBUG [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e-top 2023-02-09 22:55:42,945 INFO [StoreOpener-0e00bf05e3c10e81903e3afc1d91fc88-1] regionserver.HStore(310): Store=0e00bf05e3c10e81903e3afc1d91fc88/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:42,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:42,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:42,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 0e00bf05e3c10e81903e3afc1d91fc88; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71433856, jitterRate=0.06444740295410156}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:42,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 0e00bf05e3c10e81903e3afc1d91fc88: Running coprocessor pre-open hook at 1675983342885Writing region info on filesystem at 1675983342885Initializing all the Stores at 1675983342886 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983342886Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983342886Cleaning up temporary data from old regions at 1675983342947 (+61 ms)Cleaning up detritus from prior splits at 1675983342947Running coprocessor post-open hooks at 1675983342949 (+2 ms)Region opened successfully at 1675983342949 2023-02-09 22:55:42,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88., pid=117, masterSystemTime=1675983342882 2023-02-09 22:55:42,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:42,951 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:42,951 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:42,951 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:42,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:55:42,951 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:42,951 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1912): 0e00bf05e3c10e81903e3afc1d91fc88/cf2 is initiating minor compaction (all files) 2023-02-09 22:55:42,951 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1912): 0e00bf05e3c10e81903e3afc1d91fc88/cf1 is initiating minor compaction (all files) 2023-02-09 22:55:42,951 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0e00bf05e3c10e81903e3afc1d91fc88/cf2 in testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:42,951 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0e00bf05e3c10e81903e3afc1d91fc88/cf1 in testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:42,952 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f-top] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/.tmp, totalSize=34.9 K 2023-02-09 22:55:42,952 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58-top] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/.tmp, totalSize=34.9 K 2023-02-09 22:55:42,952 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983338028 2023-02-09 22:55:42,952 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983338028 2023-02-09 22:55:42,952 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983338201 2023-02-09 22:55:42,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:42,953 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983338201 2023-02-09 22:55:42,953 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0e00bf05e3c10e81903e3afc1d91fc88, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:42,953 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983338401 2023-02-09 22:55:42,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:42,953 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983342953"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983342953"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983342953"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983342953"}]},"ts":"1675983342953"} 2023-02-09 22:55:42,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:42,953 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983338401 2023-02-09 22:55:42,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 11c11386996225c96ffa8a1e7e902af2, NAME => 'testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 22:55:42,953 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983338594 2023-02-09 22:55:42,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:42,954 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983338594 2023-02-09 22:55:42,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:42,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:42,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:42,954 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:42,954 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675983338807 2023-02-09 22:55:42,954 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675983338807 2023-02-09 22:55:42,954 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675983338987 2023-02-09 22:55:42,954 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675983338987 2023-02-09 22:55:42,955 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983339166 2023-02-09 22:55:42,955 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983339166 2023-02-09 22:55:42,963 INFO [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:42,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-09 22:55:42,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0e00bf05e3c10e81903e3afc1d91fc88, server=jenkins-hbase9.apache.org,40413,1675983241505 in 223 msec 2023-02-09 22:55:42,964 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1 2023-02-09 22:55:42,964 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1 2023-02-09 22:55:42,965 INFO [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11c11386996225c96ffa8a1e7e902af2 columnFamilyName cf1 2023-02-09 22:55:42,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, ASSIGN in 389 msec 2023-02-09 22:55:42,967 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0e00bf05e3c10e81903e3afc1d91fc88#cf1#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:42,967 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): 0e00bf05e3c10e81903e3afc1d91fc88#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:42,978 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f-bottom 2023-02-09 22:55:42,981 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa-bottom 2023-02-09 22:55:42,982 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/.tmp/cf2/27cee891b99a4d6683ac479210f3779d as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/27cee891b99a4d6683ac479210f3779d 2023-02-09 22:55:42,983 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/.tmp/cf1/bd650a93749943eb939f77d98207c8b9 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/bd650a93749943eb939f77d98207c8b9 2023-02-09 22:55:42,984 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565-bottom 2023-02-09 22:55:42,987 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0e00bf05e3c10e81903e3afc1d91fc88/cf2 of 0e00bf05e3c10e81903e3afc1d91fc88 into 27cee891b99a4d6683ac479210f3779d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:42,987 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0e00bf05e3c10e81903e3afc1d91fc88: 2023-02-09 22:55:42,987 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88., storeName=0e00bf05e3c10e81903e3afc1d91fc88/cf2, priority=9, startTime=1675983342951; duration=0sec 2023-02-09 22:55:42,987 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:42,988 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0e00bf05e3c10e81903e3afc1d91fc88/cf1 of 0e00bf05e3c10e81903e3afc1d91fc88 into bd650a93749943eb939f77d98207c8b9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:42,988 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c-bottom 2023-02-09 22:55:42,988 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0e00bf05e3c10e81903e3afc1d91fc88: 2023-02-09 22:55:42,988 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88., storeName=0e00bf05e3c10e81903e3afc1d91fc88/cf1, priority=9, startTime=1675983342950; duration=0sec 2023-02-09 22:55:42,988 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:42,990 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c-bottom 2023-02-09 22:55:42,993 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8-bottom 2023-02-09 22:55:42,996 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b-bottom 2023-02-09 22:55:42,996 INFO [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(310): Store=11c11386996225c96ffa8a1e7e902af2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:42,996 INFO [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:42,997 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2 2023-02-09 22:55:42,997 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2 2023-02-09 22:55:42,997 INFO [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 11c11386996225c96ffa8a1e7e902af2 columnFamilyName cf2 2023-02-09 22:55:43,007 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c-bottom 2023-02-09 22:55:43,009 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286-bottom 2023-02-09 22:55:43,012 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc-bottom 2023-02-09 22:55:43,014 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58-bottom 2023-02-09 22:55:43,016 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958-bottom 2023-02-09 22:55:43,019 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb-bottom 2023-02-09 22:55:43,022 DEBUG [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e-bottom 2023-02-09 22:55:43,022 INFO [StoreOpener-11c11386996225c96ffa8a1e7e902af2-1] regionserver.HStore(310): Store=11c11386996225c96ffa8a1e7e902af2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:43,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:43,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:43,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:43,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:43,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 11c11386996225c96ffa8a1e7e902af2; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65611859, jitterRate=-0.02230711281299591}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:43,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 11c11386996225c96ffa8a1e7e902af2: Running coprocessor pre-open hook at 1675983342954Writing region info on filesystem at 1675983342954Initializing all the Stores at 1675983342955 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983342956 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983342963 (+7 ms)Cleaning up temporary data from old regions at 1675983343024 (+61 ms)Cleaning up detritus from prior splits at 1675983343024Running coprocessor post-open hooks at 1675983343027 (+3 ms)Region opened successfully at 1675983343027 2023-02-09 22:55:43,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2., pid=118, masterSystemTime=1675983342882 2023-02-09 22:55:43,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:43,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:55:43,028 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:43,028 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:43,028 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,029 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,029 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1912): 11c11386996225c96ffa8a1e7e902af2/cf1 is initiating minor compaction (all files) 2023-02-09 22:55:43,029 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1912): 11c11386996225c96ffa8a1e7e902af2/cf2 is initiating minor compaction (all files) 2023-02-09 22:55:43,029 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 11c11386996225c96ffa8a1e7e902af2/cf1 in testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,029 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 11c11386996225c96ffa8a1e7e902af2/cf2 in testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,029 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/.tmp, totalSize=34.9 K 2023-02-09 22:55:43,029 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/.tmp, totalSize=34.9 K 2023-02-09 22:55:43,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,030 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983338028 2023-02-09 22:55:43,030 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983338028 2023-02-09 22:55:43,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,030 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=11c11386996225c96ffa8a1e7e902af2, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:43,030 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983338201 2023-02-09 22:55:43,030 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983343030"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983343030"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983343030"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983343030"}]},"ts":"1675983343030"} 2023-02-09 22:55:43,031 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983338201 2023-02-09 22:55:43,031 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983338401 2023-02-09 22:55:43,031 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983338401 2023-02-09 22:55:43,031 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983338594 2023-02-09 22:55:43,032 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983338594 2023-02-09 22:55:43,032 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983338807 2023-02-09 22:55:43,032 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting 97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983338807 2023-02-09 22:55:43,032 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983338987 2023-02-09 22:55:43,033 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983338987 2023-02-09 22:55:43,033 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting 2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983339166 2023-02-09 22:55:43,033 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983339166 2023-02-09 22:55:43,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-09 22:55:43,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 11c11386996225c96ffa8a1e7e902af2, server=jenkins-hbase9.apache.org,40413,1675983241505 in 298 msec 2023-02-09 22:55:43,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-09 22:55:43,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, ASSIGN in 460 msec 2023-02-09 22:55:43,037 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 22:55:43,037 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:43,037 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 22:55:43,046 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): 11c11386996225c96ffa8a1e7e902af2#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:43,046 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): 11c11386996225c96ffa8a1e7e902af2#cf2#compaction#111 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:43,068 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/.tmp/cf1/840f0e5266914c29b585f1d3807485af as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/840f0e5266914c29b585f1d3807485af 2023-02-09 22:55:43,070 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/.tmp/cf2/4e0903cf672c48f89091d019251c526e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4e0903cf672c48f89091d019251c526e 2023-02-09 22:55:43,072 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 11c11386996225c96ffa8a1e7e902af2/cf1 of 11c11386996225c96ffa8a1e7e902af2 into 840f0e5266914c29b585f1d3807485af(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:43,072 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11c11386996225c96ffa8a1e7e902af2: 2023-02-09 22:55:43,072 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2., storeName=11c11386996225c96ffa8a1e7e902af2/cf1, priority=9, startTime=1675983343027; duration=0sec 2023-02-09 22:55:43,073 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:43,075 INFO [Listener at localhost/44743] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,075 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 22:55:43,075 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 11c11386996225c96ffa8a1e7e902af2/cf2 of 11c11386996225c96ffa8a1e7e902af2 into 4e0903cf672c48f89091d019251c526e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:43,075 INFO [Listener at localhost/44743] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:43,075 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 11c11386996225c96ffa8a1e7e902af2: 2023-02-09 22:55:43,075 INFO [Listener at localhost/44743] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:43,075 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2., storeName=11c11386996225c96ffa8a1e7e902af2/cf2, priority=9, startTime=1675983343028; duration=0sec 2023-02-09 22:55:43,075 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:43,075 ERROR [Listener at localhost/44743] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 22:55:43,076 INFO [Listener at localhost/44743] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 22:55:43,076 INFO [Listener at localhost/44743] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 22:55:43,076 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 22:55:43,076 INFO [Listener at localhost/44743] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:55:43,076 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-09 22:55:43,078 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 22:55:43,079 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 22:55:43,079 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 22:55:43,079 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:55:43,079 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:43,080 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a6ea97c1cacf1b7de3443bbe631266e3, daughterA=c4aae79c5625c8879e05e5374437a060, daughterB=f43c7c3e652b8b8ace7943731e4b0afe 2023-02-09 22:55:43,080 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:43,080 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 22:55:43,080 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:43,080 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=c0d992ad86816a23e23291a2fe7d092b, daughterA=80ed137b13594853114cbdbfcc0d5699, daughterB=c029d6f8a5817f8d8305ca00520f98be 2023-02-09 22:55:43,080 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:43,081 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 22:55:43,081 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:43,081 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f0f398cb96c5ad5c92cff9427e7b68fd, daughterA=9feae2fd1805a15b1414748d430e8cee, daughterB=25e75757713f4d54468876cc23fd3fd9 2023-02-09 22:55:43,081 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f0f398cb96c5ad5c92cff9427e7b68fd, ASSIGN 2023-02-09 22:55:43,081 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:43,082 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 22:55:43,082 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:43,082 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=57fde352bce7caf7c73c967d7b39b213, daughterA=8159207c39763434e943cce3d99a0ffb, daughterB=dbacfa106d4f2da40649a61989eade8a 2023-02-09 22:55:43,082 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:43,082 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 22:55:43,082 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:43,083 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e757cfcd8ee3a29c09867ad25a0805c2, daughterA=069696cc5e1a2ad5e9ee2aebcbea0d8a, daughterB=c115dc831efcb6b59f98b9e48a74859b 2023-02-09 22:55:43,083 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:43,083 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 22:55:43,083 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:43,083 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:43,084 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase9.apache.org,41885,1675983241428, table=testRollbackForSplitTableRegionWithReplica, region=27ab93e5042ea955274af1f95b42abeb is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=a5f8afc611ffecae749b3b56718b0e8b, daughterA=90a6f5c07ccc7c8c58a1bf6906dcfbda, daughterB=75460a0b62199d97615aabfc38e0c70f 2023-02-09 22:55:43,084 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:43,084 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 22:55:43,084 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:43,084 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:43,084 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 22:55:43,084 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,085 ERROR [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid 73c0fc0c-d116-48f6-bf3b-b31855654d02) service to localhost/127.0.0.1:41659 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, ASSIGN 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, ASSIGN 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0e00bf05e3c10e81903e3afc1d91fc88, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 11c11386996225c96ffa8a1e7e902af2, server=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:43,086 DEBUG [Listener at localhost/44743] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 held the lock before restarting, call acquireLock to restore it. 2023-02-09 22:55:43,087 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,087 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,087 INFO [Listener at localhost/44743] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,087 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:43,087 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 98929a422906bc0bcc11a422f24fe541, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:43,087 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:43,087 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:43,087 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 0e00bf05e3c10e81903e3afc1d91fc88, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:43,087 DEBUG [Listener at localhost/44743] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 11c11386996225c96ffa8a1e7e902af2, server=jenkins-hbase9.apache.org,40413,1675983241505 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 22:55:43,087 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-09 22:55:43,088 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 22:55:43,088 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473, table=hbase:meta, region=1588230740 2023-02-09 22:55:43,089 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983373089 2023-02-09 22:55:43,091 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,091 DEBUG [Listener at localhost/44743] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:55:43,094 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=f0fb98f20238a9a9ac1a9a9d1be29f13, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,41885,1675983241428, regionLocation=jenkins-hbase9.apache.org,41885,1675983241428, openSeqNum=2 2023-02-09 22:55:43,094 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=98929a422906bc0bcc11a422f24fe541, regionState=CLOSED, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=2 2023-02-09 22:55:43,094 INFO [Listener at localhost/44743] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase9.apache.org,40413,1675983241505, table=testRecoveryAndDoubleExecution, region=98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,095 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=11c11386996225c96ffa8a1e7e902af2, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=86 2023-02-09 22:55:43,095 DEBUG [Listener at localhost/44743] assignment.RegionStateStore(147): Load hbase:meta entry region=0e00bf05e3c10e81903e3afc1d91fc88, regionState=OPEN, lastHost=jenkins-hbase9.apache.org,40413,1675983241505, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505, openSeqNum=86 2023-02-09 22:55:43,095 INFO [Listener at localhost/44743] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 22:55:43,095 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983403095 2023-02-09 22:55:43,095 INFO [Listener at localhost/44743] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983463095 2023-02-09 22:55:43,095 INFO [Listener at localhost/44743] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 22:55:43,095 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 22:55:43,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=98929a422906bc0bcc11a422f24fe541, daughterA=11c11386996225c96ffa8a1e7e902af2, daughterB=0e00bf05e3c10e81903e3afc1d91fc88 in 3.6690 sec 2023-02-09 22:55:43,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40413] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.2.10:47988 deadline: 1675983403346, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541. is not online on jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:43,460 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 22:55:43,467 INFO [Listener at localhost/44743] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 22:55:43,469 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 22:55:43,479 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:43,481 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-09 22:55:43,481 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testRecoveryAndDoubleExecution 2023-02-09 22:55:43,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 22:55:43,484 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983343484"}]},"ts":"1675983343484"} 2023-02-09 22:55:43,485 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 22:55:43,487 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-09 22:55:43,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, UNASSIGN}] 2023-02-09 22:55:43,488 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, UNASSIGN 2023-02-09 22:55:43,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0e00bf05e3c10e81903e3afc1d91fc88, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:43,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983343489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983343489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983343489"}]},"ts":"1675983343489"} 2023-02-09 22:55:43,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 0e00bf05e3c10e81903e3afc1d91fc88, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:43,491 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, UNASSIGN 2023-02-09 22:55:43,492 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=11c11386996225c96ffa8a1e7e902af2, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:43,492 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983343492"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983343492"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983343492"}]},"ts":"1675983343492"} 2023-02-09 22:55:43,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 11c11386996225c96ffa8a1e7e902af2, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:43,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 22:55:43,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 0e00bf05e3c10e81903e3afc1d91fc88, disabling compactions & flushes 2023-02-09 22:55:43,640 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:43,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:43,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. after waiting 0 ms 2023-02-09 22:55:43,640 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:43,641 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f-top] to archive 2023-02-09 22:55:43,642 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:43,643 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,644 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,645 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,646 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,647 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,648 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,649 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,650 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58-top] to archive 2023-02-09 22:55:43,650 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:43,651 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,652 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,653 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,654 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,655 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,656 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,657 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:55:43,660 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:43,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88. 2023-02-09 22:55:43,661 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 0e00bf05e3c10e81903e3afc1d91fc88: Waiting for close lock at 1675983343640Running coprocessor pre-close hooks at 1675983343640Disabling compacts and flushes for region at 1675983343640Disabling writes for close at 1675983343640Writing region close event to WAL at 1675983343658 (+18 ms)Running coprocessor post-close hooks at 1675983343660 (+2 ms)Closed at 1675983343661 (+1 ms) 2023-02-09 22:55:43,663 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,663 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:43,663 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 11c11386996225c96ffa8a1e7e902af2, disabling compactions & flushes 2023-02-09 22:55:43,663 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,663 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,663 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. after waiting 0 ms 2023-02-09 22:55:43,663 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,663 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0e00bf05e3c10e81903e3afc1d91fc88, regionState=CLOSED 2023-02-09 22:55:43,663 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983343663"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983343663"}]},"ts":"1675983343663"} 2023-02-09 22:55:43,664 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f-bottom] to archive 2023-02-09 22:55:43,664 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:43,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-09 22:55:43,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 0e00bf05e3c10e81903e3afc1d91fc88, server=jenkins-hbase9.apache.org,40413,1675983241505 in 174 msec 2023-02-09 22:55:43,666 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0e00bf05e3c10e81903e3afc1d91fc88, UNASSIGN in 179 msec 2023-02-09 22:55:43,667 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,668 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,669 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,669 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,670 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,671 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,672 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58-bottom] to archive 2023-02-09 22:55:43,673 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:43,674 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,675 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,675 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,676 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,677 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,678 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,679 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:55:43,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:43,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2. 2023-02-09 22:55:43,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 11c11386996225c96ffa8a1e7e902af2: Waiting for close lock at 1675983343663Running coprocessor pre-close hooks at 1675983343663Disabling compacts and flushes for region at 1675983343663Disabling writes for close at 1675983343663Writing region close event to WAL at 1675983343679 (+16 ms)Running coprocessor post-close hooks at 1675983343682 (+3 ms)Closed at 1675983343682 2023-02-09 22:55:43,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:43,684 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=11c11386996225c96ffa8a1e7e902af2, regionState=CLOSED 2023-02-09 22:55:43,684 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983343684"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983343684"}]},"ts":"1675983343684"} 2023-02-09 22:55:43,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-09 22:55:43,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 11c11386996225c96ffa8a1e7e902af2, server=jenkins-hbase9.apache.org,40413,1675983241505 in 192 msec 2023-02-09 22:55:43,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-09 22:55:43,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=11c11386996225c96ffa8a1e7e902af2, UNASSIGN in 199 msec 2023-02-09 22:55:43,688 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983343688"}]},"ts":"1675983343688"} 2023-02-09 22:55:43,689 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 22:55:43,690 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-09 22:55:43,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 210 msec 2023-02-09 22:55:43,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 22:55:43,786 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-09 22:55:43,787 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testRecoveryAndDoubleExecution 2023-02-09 22:55:43,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,790 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 22:55:43,790 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,795 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,795 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:43,795 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,797 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/recovered.edits] 2023-02-09 22:55:43,797 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/recovered.edits] 2023-02-09 22:55:43,797 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/recovered.edits] 2023-02-09 22:55:43,801 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/.regioninfo to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/.regioninfo 2023-02-09 22:55:43,801 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/.regioninfo to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/.regioninfo 2023-02-09 22:55:43,802 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/2ca6670211d343e8be0ecde51c86375f 2023-02-09 22:55:43,804 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa 2023-02-09 22:55:43,804 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,804 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/2ca6670211d343e8be0ecde51c86375f.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,805 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/899d6c7c2db04e51867e1ad6ae79e565 2023-02-09 22:55:43,805 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,805 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/7e1cd234c5194b1b9ab58754fa2c8cfa.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,806 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/d6f1c50f1504489b8a0bcf85c56c836c 2023-02-09 22:55:43,807 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,807 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/899d6c7c2db04e51867e1ad6ae79e565.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,808 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/ef378bcd97e641fc8d71a701fe9eec4c 2023-02-09 22:55:43,808 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,809 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/d6f1c50f1504489b8a0bcf85c56c836c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,809 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f0e1437a3cdb4a5397730806670697c8 2023-02-09 22:55:43,810 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,810 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/ef378bcd97e641fc8d71a701fe9eec4c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,810 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf1/f4eaa7094af744cf829be1bcedcb0a8b 2023-02-09 22:55:43,811 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,811 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f0e1437a3cdb4a5397730806670697c8.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,818 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,819 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf1/f4eaa7094af744cf829be1bcedcb0a8b.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,820 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/3f8a5db51e7a4a12bd090effb82a931c 2023-02-09 22:55:43,821 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/4fd3d2ca44cc4d239bb0f6780b050286 2023-02-09 22:55:43,822 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,823 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/3f8a5db51e7a4a12bd090effb82a931c.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,824 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/97e42b893c994c21896bc4210054edcc 2023-02-09 22:55:43,824 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,824 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/4fd3d2ca44cc4d239bb0f6780b050286.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,825 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,825 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/a0bcce43db68456f900fc90ad62b2f58 2023-02-09 22:55:43,826 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/97e42b893c994c21896bc4210054edcc.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,826 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,827 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/c674402dad904a228c16b34bc1d36958 2023-02-09 22:55:43,827 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/a0bcce43db68456f900fc90ad62b2f58.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,828 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,828 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/cd606858ec664914b5c555b9bf13d9cb 2023-02-09 22:55:43,828 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/c674402dad904a228c16b34bc1d36958.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,829 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/cf2/db062cdb5c8044eca8ebf5c914954a1e 2023-02-09 22:55:43,829 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,832 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/cd606858ec664914b5c555b9bf13d9cb.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,832 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/11c11386996225c96ffa8a1e7e902af2/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,835 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/0e00bf05e3c10e81903e3afc1d91fc88/cf2/db062cdb5c8044eca8ebf5c914954a1e.98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,835 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/recovered.edits/85.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541/recovered.edits/85.seqid 2023-02-09 22:55:43,835 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/98929a422906bc0bcc11a422f24fe541 2023-02-09 22:55:43,839 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/840f0e5266914c29b585f1d3807485af to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf1/840f0e5266914c29b585f1d3807485af 2023-02-09 22:55:43,840 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/bd650a93749943eb939f77d98207c8b9 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf1/bd650a93749943eb939f77d98207c8b9 2023-02-09 22:55:43,842 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4e0903cf672c48f89091d019251c526e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/cf2/4e0903cf672c48f89091d019251c526e 2023-02-09 22:55:43,842 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/27cee891b99a4d6683ac479210f3779d to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/cf2/27cee891b99a4d6683ac479210f3779d 2023-02-09 22:55:43,844 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2/recovered.edits/90.seqid 2023-02-09 22:55:43,845 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/11c11386996225c96ffa8a1e7e902af2 2023-02-09 22:55:43,845 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88/recovered.edits/90.seqid 2023-02-09 22:55:43,845 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testRecoveryAndDoubleExecution/0e00bf05e3c10e81903e3afc1d91fc88 2023-02-09 22:55:43,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 22:55:43,848 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,849 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-09 22:55:43,851 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-09 22:55:43,852 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,852 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-09 22:55:43,852 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983343852"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:43,852 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983343852"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:43,852 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983343852"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:43,854 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:55:43,854 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 98929a422906bc0bcc11a422f24fe541, NAME => 'testRecoveryAndDoubleExecution,,1675983337525.98929a422906bc0bcc11a422f24fe541.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 11c11386996225c96ffa8a1e7e902af2, NAME => 'testRecoveryAndDoubleExecution,,1675983339426.11c11386996225c96ffa8a1e7e902af2.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0e00bf05e3c10e81903e3afc1d91fc88, NAME => 'testRecoveryAndDoubleExecution,41,1675983339426.0e00bf05e3c10e81903e3afc1d91fc88.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 22:55:43,854 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-09 22:55:43,854 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983343854"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:43,855 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-09 22:55:43,857 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 22:55:43,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 70 msec 2023-02-09 22:55:43,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 22:55:43,892 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-09 22:55:43,907 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 368) - Thread LEAK? -, OpenFileDescriptor=840 (was 745) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=120 (was 104) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=10641 (was 10676) 2023-02-09 22:55:43,923 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=840, MaxFileDescriptor=60000, SystemLoadAverage=120, ProcessCount=168, AvailableMemoryMB=10640 2023-02-09 22:55:43,923 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:43,927 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:55:43,927 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:55:43,928 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:43,929 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:43,930 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:43,931 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:43,932 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0 empty. 2023-02-09 22:55:43,932 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:43,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 22:55:43,941 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:43,942 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3c1463347f30b7be526325deae6a0aa0, NAME => 'testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:55:43,948 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:43,948 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 3c1463347f30b7be526325deae6a0aa0, disabling compactions & flushes 2023-02-09 22:55:43,948 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:43,948 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:43,948 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. after waiting 0 ms 2023-02-09 22:55:43,948 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:43,949 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:43,949 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 3c1463347f30b7be526325deae6a0aa0: Waiting for close lock at 1675983343948Disabling compacts and flushes for region at 1675983343948Disabling writes for close at 1675983343948Writing region close event to WAL at 1675983343948Closed at 1675983343948 2023-02-09 22:55:43,950 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:43,951 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983343951"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983343951"}]},"ts":"1675983343951"} 2023-02-09 22:55:43,952 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:55:43,953 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:43,953 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983343953"}]},"ts":"1675983343953"} 2023-02-09 22:55:43,954 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-09 22:55:43,957 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:55:43,958 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:55:43,958 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:55:43,958 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:55:43,958 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:55:43,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3c1463347f30b7be526325deae6a0aa0, ASSIGN}] 2023-02-09 22:55:43,959 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3c1463347f30b7be526325deae6a0aa0, ASSIGN 2023-02-09 22:55:43,960 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3c1463347f30b7be526325deae6a0aa0, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:55:44,080 ERROR [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid 4256d769-ee79-40f3-a328-ac20eb1446de) service to localhost/127.0.0.1:41659 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:55:44,110 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:55:44,111 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=3c1463347f30b7be526325deae6a0aa0, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,111 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983344111"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983344111"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983344111"}]},"ts":"1675983344111"} 2023-02-09 22:55:44,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 3c1463347f30b7be526325deae6a0aa0, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:44,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:44,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3c1463347f30b7be526325deae6a0aa0, NAME => 'testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:44,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:44,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:44,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,268 INFO [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,269 DEBUG [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1 2023-02-09 22:55:44,269 DEBUG [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1 2023-02-09 22:55:44,269 INFO [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c1463347f30b7be526325deae6a0aa0 columnFamilyName cf1 2023-02-09 22:55:44,269 INFO [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] regionserver.HStore(310): Store=3c1463347f30b7be526325deae6a0aa0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:44,270 INFO [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,270 DEBUG [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2 2023-02-09 22:55:44,270 DEBUG [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2 2023-02-09 22:55:44,271 INFO [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3c1463347f30b7be526325deae6a0aa0 columnFamilyName cf2 2023-02-09 22:55:44,271 INFO [StoreOpener-3c1463347f30b7be526325deae6a0aa0-1] regionserver.HStore(310): Store=3c1463347f30b7be526325deae6a0aa0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:44,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:44,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:44,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:44,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 3c1463347f30b7be526325deae6a0aa0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63193174, jitterRate=-0.05834832787513733}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:44,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 3c1463347f30b7be526325deae6a0aa0: Running coprocessor pre-open hook at 1675983344267Writing region info on filesystem at 1675983344267Initializing all the Stores at 1675983344267Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983344267Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983344268 (+1 ms)Cleaning up temporary data from old regions at 1675983344272 (+4 ms)Cleaning up detritus from prior splits at 1675983344273 (+1 ms)Running coprocessor post-open hooks at 1675983344276 (+3 ms)Region opened successfully at 1675983344276 2023-02-09 22:55:44,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0., pid=127, masterSystemTime=1675983344264 2023-02-09 22:55:44,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:44,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:44,278 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=3c1463347f30b7be526325deae6a0aa0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,278 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983344278"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983344278"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983344278"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983344278"}]},"ts":"1675983344278"} 2023-02-09 22:55:44,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-09 22:55:44,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 3c1463347f30b7be526325deae6a0aa0, server=jenkins-hbase9.apache.org,40413,1675983241505 in 168 msec 2023-02-09 22:55:44,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-09 22:55:44,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3c1463347f30b7be526325deae6a0aa0, ASSIGN in 323 msec 2023-02-09 22:55:44,283 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:44,283 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983344283"}]},"ts":"1675983344283"} 2023-02-09 22:55:44,284 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-09 22:55:44,286 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:44,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 358 msec 2023-02-09 22:55:44,436 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:44,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,438 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:44,438 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,438 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 22:55:44,438 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 22:55:44,438 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,438 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,440 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,440 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,440 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,440 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,441 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,441 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,441 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:44,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:44,442 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:44,443 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,443 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:44,443 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,443 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:44,443 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:44,443 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:44,443 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:44,443 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. started... 2023-02-09 22:55:44,444 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3c1463347f30b7be526325deae6a0aa0 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:55:44,444 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,445 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,445 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,445 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:44,446 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,446 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:44,452 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/f44b9828c98c4f72ad2986fed4ee3134 2023-02-09 22:55:44,462 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/26b984b169d248d984ee5f9123374416 2023-02-09 22:55:44,465 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/f44b9828c98c4f72ad2986fed4ee3134 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/f44b9828c98c4f72ad2986fed4ee3134 2023-02-09 22:55:44,468 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/f44b9828c98c4f72ad2986fed4ee3134, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:44,469 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/26b984b169d248d984ee5f9123374416 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/26b984b169d248d984ee5f9123374416 2023-02-09 22:55:44,472 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/26b984b169d248d984ee5f9123374416, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:44,472 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 28ms, sequenceid=6, compaction requested=false 2023-02-09 22:55:44,472 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:44,473 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3c1463347f30b7be526325deae6a0aa0: 2023-02-09 22:55:44,473 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:44,473 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:44,473 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:44,473 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,473 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:44,473 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:44,474 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,474 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:44,475 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,475 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:44,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:44,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 22:55:44,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b175693[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:44,477 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 22:55:44,477 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,479 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:44,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:44,479 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:44,480 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:44,481 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:44,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:44,482 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:44,482 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:44,482 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:44,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:55:44,484 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 22:55:44,484 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:44,484 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:44,484 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,484 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:44,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:44,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:44,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:44,486 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:44,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:44,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:44,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:44,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:44,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,495 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,495 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,495 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,496 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:44,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:44,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,496 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:44,496 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:44,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:44,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:44,496 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:44,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:44,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,497 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:44,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:44,597 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:44,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:44,607 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:44,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:44,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,608 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:44,609 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,609 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 22:55:44,609 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 22:55:44,609 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,609 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,612 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,612 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,612 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,612 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:44,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,612 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:44,612 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:44,613 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:44,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:44,613 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,613 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:44,613 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:44,613 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,613 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:44,613 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:44,614 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:44,614 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:44,614 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. started... 2023-02-09 22:55:44,614 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3c1463347f30b7be526325deae6a0aa0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:44,615 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,617 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,617 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:44,617 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,617 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:44,617 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,022 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/d6416cc775bd443fa661a73dd36d5a39 2023-02-09 22:55:45,433 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/9ace2b3927fa40e0844f5ae99b3c53e9 2023-02-09 22:55:45,437 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/d6416cc775bd443fa661a73dd36d5a39 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/d6416cc775bd443fa661a73dd36d5a39 2023-02-09 22:55:45,440 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/d6416cc775bd443fa661a73dd36d5a39, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:45,440 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/9ace2b3927fa40e0844f5ae99b3c53e9 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/9ace2b3927fa40e0844f5ae99b3c53e9 2023-02-09 22:55:45,443 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/9ace2b3927fa40e0844f5ae99b3c53e9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:45,443 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 829ms, sequenceid=19, compaction requested=false 2023-02-09 22:55:45,444 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3c1463347f30b7be526325deae6a0aa0: 2023-02-09 22:55:45,444 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:45,444 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:45,444 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:45,444 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,444 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,444 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,446 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,446 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,447 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,447 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 22:55:45,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1cb65915[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:45,449 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 22:55:45,449 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,450 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,450 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:45,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:45,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,450 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:45,453 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,453 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,453 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,454 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,454 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,454 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,454 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,454 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,454 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,454 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:45,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:55:45,456 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 22:55:45,456 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:45,456 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:45,456 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,456 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:45,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,458 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,459 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,459 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,461 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,461 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,461 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,463 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,463 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:45,464 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:45,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,463 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,464 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,464 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,464 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,464 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,464 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,464 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:45,464 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,465 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,465 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,465 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,465 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,565 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:45,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:45,575 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:45,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:45,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,577 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:45,577 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,577 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 22:55:45,577 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 22:55:45,577 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,577 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,580 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,580 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,580 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,580 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,580 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,580 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:45,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,581 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,581 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:45,582 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:45,582 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:45,582 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. started... 2023-02-09 22:55:45,582 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3c1463347f30b7be526325deae6a0aa0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:45,584 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,584 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,584 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,584 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,584 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,584 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,591 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/4ae0acc237224157aa55e5f9b30fdada 2023-02-09 22:55:45,600 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e 2023-02-09 22:55:45,603 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/4ae0acc237224157aa55e5f9b30fdada as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/4ae0acc237224157aa55e5f9b30fdada 2023-02-09 22:55:45,606 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/4ae0acc237224157aa55e5f9b30fdada, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:45,607 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e 2023-02-09 22:55:45,609 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:45,610 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 28ms, sequenceid=32, compaction requested=true 2023-02-09 22:55:45,610 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3c1463347f30b7be526325deae6a0aa0: 2023-02-09 22:55:45,610 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:45,610 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:45,610 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:45,610 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,610 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,610 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,612 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,612 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,612 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,612 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 22:55:45,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61a94892[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:45,614 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 22:55:45,614 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,615 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,615 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,616 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:45,616 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:45,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,622 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,622 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,623 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,623 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,623 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:45,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:55:45,625 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 22:55:45,625 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:45,625 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:45,625 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,625 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:45,627 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,627 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,627 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,627 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,627 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,628 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,628 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,630 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,630 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,630 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,634 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,634 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,634 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,634 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,634 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,635 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:45,635 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:45,635 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,635 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,636 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,636 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,636 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,636 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,636 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,735 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:45,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:45,747 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:45,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:45,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,748 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:45,748 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,748 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 22:55:45,748 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 22:55:45,749 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,749 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,751 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,751 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,751 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,751 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:45,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,752 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,753 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,753 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,753 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,753 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,754 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,754 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,754 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,754 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:45,754 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:45,754 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:45,754 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. started... 2023-02-09 22:55:45,754 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3c1463347f30b7be526325deae6a0aa0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:45,755 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,756 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,756 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,756 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,756 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,756 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,764 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/aaefeb9a6154455bb1a33da5617552e9 2023-02-09 22:55:45,773 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/47cdd6751dcd491c91910a7b0ddb8fcb 2023-02-09 22:55:45,776 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/aaefeb9a6154455bb1a33da5617552e9 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/aaefeb9a6154455bb1a33da5617552e9 2023-02-09 22:55:45,779 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/aaefeb9a6154455bb1a33da5617552e9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:45,780 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/47cdd6751dcd491c91910a7b0ddb8fcb as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/47cdd6751dcd491c91910a7b0ddb8fcb 2023-02-09 22:55:45,783 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/47cdd6751dcd491c91910a7b0ddb8fcb, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:45,784 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 30ms, sequenceid=45, compaction requested=true 2023-02-09 22:55:45,784 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3c1463347f30b7be526325deae6a0aa0: 2023-02-09 22:55:45,784 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:45,784 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:45,784 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:45,784 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,784 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,784 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,787 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,787 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,787 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,787 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 22:55:45,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b42c29b[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:45,790 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 22:55:45,790 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,792 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,792 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:45,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,792 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,794 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,794 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:45,797 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:55:45,797 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 22:55:45,797 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:45,797 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:45,797 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,797 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:45,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,799 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,799 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,800 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,800 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,802 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,808 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:45,809 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:45,808 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,808 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:45,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,809 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,810 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,810 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,810 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,909 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:45,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:45,920 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:55:45,920 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:45,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:45,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,922 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:45,922 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,922 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 22:55:45,922 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 22:55:45,922 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,922 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,924 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,924 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,924 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,924 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,924 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,924 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:45,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,925 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,927 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,927 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,927 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:45,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:45,931 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:45,931 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:45,931 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. started... 2023-02-09 22:55:45,931 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3c1463347f30b7be526325deae6a0aa0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:45,932 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,933 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,933 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,945 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/5c8e0dea26f143b789f63823450e6265 2023-02-09 22:55:45,961 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/71d18d647b54448d83251165f2580ca8 2023-02-09 22:55:45,964 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/5c8e0dea26f143b789f63823450e6265 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/5c8e0dea26f143b789f63823450e6265 2023-02-09 22:55:45,967 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/5c8e0dea26f143b789f63823450e6265, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:45,968 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/71d18d647b54448d83251165f2580ca8 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/71d18d647b54448d83251165f2580ca8 2023-02-09 22:55:45,970 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/71d18d647b54448d83251165f2580ca8, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:45,971 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 40ms, sequenceid=58, compaction requested=true 2023-02-09 22:55:45,971 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3c1463347f30b7be526325deae6a0aa0: 2023-02-09 22:55:45,971 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:45,971 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:45,971 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:45,971 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,971 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:45,971 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:45,974 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,974 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,974 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,974 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 22:55:45,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6aa0baae[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:45,976 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 22:55:45,976 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,977 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,977 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,977 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,978 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,978 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:45,978 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:45,979 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,979 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,979 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,979 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,980 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,980 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,980 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:45,980 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,980 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,980 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:45,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:55:45,983 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 22:55:45,983 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:45,983 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:45,983 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,983 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:45,986 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,986 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,986 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,986 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,987 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,988 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,988 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,989 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,991 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:45,991 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:45,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,991 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,991 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:45,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:45,992 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,092 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:46,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:46,103 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:46,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:46,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,104 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:46,104 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:46,104 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 22:55:46,104 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 22:55:46,104 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,104 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,105 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:46,105 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,105 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:46,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:46,106 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:46,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:46,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:46,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:46,106 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,106 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:46,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:46,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:46,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:46,107 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:46,107 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,107 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:46,107 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:46,107 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:46,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 22:55:46,107 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:46,108 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:46,108 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:46,108 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:46,108 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. started... 2023-02-09 22:55:46,108 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3c1463347f30b7be526325deae6a0aa0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:46,109 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,111 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,111 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,111 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:46,111 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,111 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:46,116 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/55e40d05dda14c20a40f5209a3006909 2023-02-09 22:55:46,126 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/4605288aa5b84830a2a2948ae89a7be0 2023-02-09 22:55:46,129 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/55e40d05dda14c20a40f5209a3006909 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/55e40d05dda14c20a40f5209a3006909 2023-02-09 22:55:46,131 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/55e40d05dda14c20a40f5209a3006909, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:46,132 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/4605288aa5b84830a2a2948ae89a7be0 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/4605288aa5b84830a2a2948ae89a7be0 2023-02-09 22:55:46,134 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/4605288aa5b84830a2a2948ae89a7be0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:46,135 INFO [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 27ms, sequenceid=71, compaction requested=true 2023-02-09 22:55:46,135 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3c1463347f30b7be526325deae6a0aa0: 2023-02-09 22:55:46,135 DEBUG [rs(jenkins-hbase9.apache.org,40413,1675983241505)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:46,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:46,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:46,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 22:55:46,135 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 22:55:46,137 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,137 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:46,137 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,137 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:46,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 22:55:46,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4470281e[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:46,139 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 22:55:46,139 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,147 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:46,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:46,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:46,147 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:46,150 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:46,150 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:46,150 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:46,151 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,151 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:46,151 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 22:55:46,151 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:46,151 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:46,151 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:46,151 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 22:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:46,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase9.apache.org,40413,1675983241505': 2023-02-09 22:55:46,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 22:55:46,153 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 22:55:46,153 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:46,153 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:46,153 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,153 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:46,155 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,155 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,155 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:46,155 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,155 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,155 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:46,156 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:46,155 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:46,156 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:46,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:46,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:46,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,158 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,158 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,158 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:46,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:46,161 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 22:55:46,161 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:46,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:46,161 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:46,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:46,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:46,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:46,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:46,162 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:46,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:46,161 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:46,162 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:46,162 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:46,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:46,162 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:46,162 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,162 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,162 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 22:55:46,262 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:46,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:46,269 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,270 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3c1463347f30b7be526325deae6a0aa0, daughterA=6eacad5bba92e31c8debd2219e721254, daughterB=9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,271 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3c1463347f30b7be526325deae6a0aa0, daughterA=6eacad5bba92e31c8debd2219e721254, daughterB=9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,271 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3c1463347f30b7be526325deae6a0aa0, daughterA=6eacad5bba92e31c8debd2219e721254, daughterB=9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,271 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3c1463347f30b7be526325deae6a0aa0, daughterA=6eacad5bba92e31c8debd2219e721254, daughterB=9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3c1463347f30b7be526325deae6a0aa0, UNASSIGN}] 2023-02-09 22:55:46,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3c1463347f30b7be526325deae6a0aa0, UNASSIGN 2023-02-09 22:55:46,273 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=3c1463347f30b7be526325deae6a0aa0, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,273 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983346273"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983346273"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983346273"}]},"ts":"1675983346273"} 2023-02-09 22:55:46,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 3c1463347f30b7be526325deae6a0aa0, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:46,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 3c1463347f30b7be526325deae6a0aa0, disabling compactions & flushes 2023-02-09 22:55:46,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:46,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:46,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. after waiting 0 ms 2023-02-09 22:55:46,426 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:46,426 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing 3c1463347f30b7be526325deae6a0aa0 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 22:55:46,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/3c5a1c55c0654a33931974cef314bc88 2023-02-09 22:55:46,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/a1698bb0cedf4eb2a4ed9242b4fce527 2023-02-09 22:55:46,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf1/3c5a1c55c0654a33931974cef314bc88 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/3c5a1c55c0654a33931974cef314bc88 2023-02-09 22:55:46,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/3c5a1c55c0654a33931974cef314bc88, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:46,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.tmp/cf2/a1698bb0cedf4eb2a4ed9242b4fce527 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/a1698bb0cedf4eb2a4ed9242b4fce527 2023-02-09 22:55:46,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/a1698bb0cedf4eb2a4ed9242b4fce527, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:46,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 29ms, sequenceid=82, compaction requested=true 2023-02-09 22:55:46,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 22:55:46,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:46,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. 2023-02-09 22:55:46,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 3c1463347f30b7be526325deae6a0aa0: Waiting for close lock at 1675983346426Running coprocessor pre-close hooks at 1675983346426Disabling compacts and flushes for region at 1675983346426Disabling writes for close at 1675983346426Obtaining lock to block concurrent updates at 1675983346426Preparing flush snapshotting stores in 3c1463347f30b7be526325deae6a0aa0 at 1675983346426Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675983346426Flushing stores of testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. at 1675983346427 (+1 ms)Flushing 3c1463347f30b7be526325deae6a0aa0/cf1: creating writer at 1675983346429 (+2 ms)Flushing 3c1463347f30b7be526325deae6a0aa0/cf1: appending metadata at 1675983346431 (+2 ms)Flushing 3c1463347f30b7be526325deae6a0aa0/cf1: closing flushed file at 1675983346431Flushing 3c1463347f30b7be526325deae6a0aa0/cf2: creating writer at 1675983346439 (+8 ms)Flushing 3c1463347f30b7be526325deae6a0aa0/cf2: appending metadata at 1675983346440 (+1 ms)Flushing 3c1463347f30b7be526325deae6a0aa0/cf2: closing flushed file at 1675983346441 (+1 ms)Flushing 3c1463347f30b7be526325deae6a0aa0/cf1: reopening flushed file at 1675983346449 (+8 ms)Flushing 3c1463347f30b7be526325deae6a0aa0/cf2: reopening flushed file at 1675983346452 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3c1463347f30b7be526325deae6a0aa0 in 29ms, sequenceid=82, compaction requested=true at 1675983346456 (+4 ms)Writing region close event to WAL at 1675983346456Running coprocessor post-close hooks at 1675983346459 (+3 ms)Closed at 1675983346460 (+1 ms) 2023-02-09 22:55:46,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=3c1463347f30b7be526325deae6a0aa0, regionState=CLOSED 2023-02-09 22:55:46,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983346461"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983346461"}]},"ts":"1675983346461"} 2023-02-09 22:55:46,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-09 22:55:46,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 3c1463347f30b7be526325deae6a0aa0, server=jenkins-hbase9.apache.org,40413,1675983241505 in 189 msec 2023-02-09 22:55:46,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-09 22:55:46,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3c1463347f30b7be526325deae6a0aa0, UNASSIGN in 192 msec 2023-02-09 22:55:46,473 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=3c1463347f30b7be526325deae6a0aa0, threads=14 2023-02-09 22:55:46,473 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/26b984b169d248d984ee5f9123374416 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,473 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/4605288aa5b84830a2a2948ae89a7be0 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,473 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/47cdd6751dcd491c91910a7b0ddb8fcb for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,473 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/71d18d647b54448d83251165f2580ca8 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/9ace2b3927fa40e0844f5ae99b3c53e9 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/a1698bb0cedf4eb2a4ed9242b4fce527 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/3c5a1c55c0654a33931974cef314bc88 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/4ae0acc237224157aa55e5f9b30fdada for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/55e40d05dda14c20a40f5209a3006909 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/5c8e0dea26f143b789f63823450e6265 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/aaefeb9a6154455bb1a33da5617552e9 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,474 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/d6416cc775bd443fa661a73dd36d5a39 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,475 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/f44b9828c98c4f72ad2986fed4ee3134 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,477 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/a1698bb0cedf4eb2a4ed9242b4fce527, top=false 2023-02-09 22:55:46,478 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/3c5a1c55c0654a33931974cef314bc88, top=false 2023-02-09 22:55:46,479 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/f44b9828c98c4f72ad2986fed4ee3134, top=false 2023-02-09 22:55:46,484 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f44b9828c98c4f72ad2986fed4ee3134 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,486 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/26b984b169d248d984ee5f9123374416, top=false 2023-02-09 22:55:46,486 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/4ae0acc237224157aa55e5f9b30fdada, top=false 2023-02-09 22:55:46,487 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/71d18d647b54448d83251165f2580ca8, top=false 2023-02-09 22:55:46,488 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/55e40d05dda14c20a40f5209a3006909, top=false 2023-02-09 22:55:46,488 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/47cdd6751dcd491c91910a7b0ddb8fcb, top=false 2023-02-09 22:55:46,490 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/4605288aa5b84830a2a2948ae89a7be0, top=false 2023-02-09 22:55:46,491 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/aaefeb9a6154455bb1a33da5617552e9, top=false 2023-02-09 22:55:46,491 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/9ace2b3927fa40e0844f5ae99b3c53e9, top=false 2023-02-09 22:55:46,491 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/f44b9828c98c4f72ad2986fed4ee3134 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,492 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e, top=false 2023-02-09 22:55:46,493 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/d6416cc775bd443fa661a73dd36d5a39, top=false 2023-02-09 22:55:46,502 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/5c8e0dea26f143b789f63823450e6265, top=false 2023-02-09 22:55:46,507 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-3c5a1c55c0654a33931974cef314bc88 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,509 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/3c5a1c55c0654a33931974cef314bc88 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,519 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f3f5e7d5ca3e4c81ae5d3f489cbcc75e for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,519 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-47cdd6751dcd491c91910a7b0ddb8fcb for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,519 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4605288aa5b84830a2a2948ae89a7be0 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,519 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-9ace2b3927fa40e0844f5ae99b3c53e9 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,520 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4ae0acc237224157aa55e5f9b30fdada for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,522 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/4ae0acc237224157aa55e5f9b30fdada for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,525 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,526 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/47cdd6751dcd491c91910a7b0ddb8fcb for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,526 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/4605288aa5b84830a2a2948ae89a7be0 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,526 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/9ace2b3927fa40e0844f5ae99b3c53e9 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,530 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-26b984b169d248d984ee5f9123374416 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,530 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-71d18d647b54448d83251165f2580ca8 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,530 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-55e40d05dda14c20a40f5209a3006909 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,531 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-a1698bb0cedf4eb2a4ed9242b4fce527 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,531 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-d6416cc775bd443fa661a73dd36d5a39 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,531 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-aaefeb9a6154455bb1a33da5617552e9 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,532 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/.splits/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-5c8e0dea26f143b789f63823450e6265 for child: 6eacad5bba92e31c8debd2219e721254, parent: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,534 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/26b984b169d248d984ee5f9123374416 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,536 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/71d18d647b54448d83251165f2580ca8 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,536 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/aaefeb9a6154455bb1a33da5617552e9 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,536 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/55e40d05dda14c20a40f5209a3006909 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,536 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/a1698bb0cedf4eb2a4ed9242b4fce527 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,536 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/d6416cc775bd443fa661a73dd36d5a39 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,539 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/5c8e0dea26f143b789f63823450e6265 for region: 3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:46,540 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 3c1463347f30b7be526325deae6a0aa0 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-09 22:55:46,570 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:55:46,571 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:55:46,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675983346573"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675983346573"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675983346573"}]},"ts":"1675983346573"} 2023-02-09 22:55:46,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983346573"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983346573"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983346573"}]},"ts":"1675983346573"} 2023-02-09 22:55:46,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983346573"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983346573"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983346573"}]},"ts":"1675983346573"} 2023-02-09 22:55:46,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6eacad5bba92e31c8debd2219e721254, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9813e50bcd0989a84947ee998d75c224, ASSIGN}] 2023-02-09 22:55:46,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9813e50bcd0989a84947ee998d75c224, ASSIGN 2023-02-09 22:55:46,577 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9813e50bcd0989a84947ee998d75c224, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:55:46,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6eacad5bba92e31c8debd2219e721254, ASSIGN 2023-02-09 22:55:46,578 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6eacad5bba92e31c8debd2219e721254, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,40413,1675983241505; forceNewPlan=false, retain=false 2023-02-09 22:55:46,727 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 22:55:46,729 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=9813e50bcd0989a84947ee998d75c224, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983346729"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983346729"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983346729"}]},"ts":"1675983346729"} 2023-02-09 22:55:46,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 9813e50bcd0989a84947ee998d75c224, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:46,732 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6eacad5bba92e31c8debd2219e721254, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,732 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983346732"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983346732"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983346732"}]},"ts":"1675983346732"} 2023-02-09 22:55:46,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 6eacad5bba92e31c8debd2219e721254, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:46,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. 2023-02-09 22:55:46,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9813e50bcd0989a84947ee998d75c224, NAME => 'testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.', STARTKEY => '71', ENDKEY => ''} 2023-02-09 22:55:46,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:46,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:46,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,885 INFO [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,886 DEBUG [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/cf1 2023-02-09 22:55:46,887 DEBUG [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/cf1 2023-02-09 22:55:46,887 INFO [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9813e50bcd0989a84947ee998d75c224 columnFamilyName cf1 2023-02-09 22:55:46,887 INFO [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] regionserver.HStore(310): Store=9813e50bcd0989a84947ee998d75c224/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:46,887 INFO [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,888 DEBUG [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/cf2 2023-02-09 22:55:46,888 DEBUG [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/cf2 2023-02-09 22:55:46,889 INFO [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9813e50bcd0989a84947ee998d75c224 columnFamilyName cf2 2023-02-09 22:55:46,889 INFO [StoreOpener-9813e50bcd0989a84947ee998d75c224-1] regionserver.HStore(310): Store=9813e50bcd0989a84947ee998d75c224/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:46,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:46,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:46,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 9813e50bcd0989a84947ee998d75c224; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67124707, jitterRate=2.3607909679412842E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:46,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 9813e50bcd0989a84947ee998d75c224: Running coprocessor pre-open hook at 1675983346885Writing region info on filesystem at 1675983346885Initializing all the Stores at 1675983346885Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983346885Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983346885Cleaning up temporary data from old regions at 1675983346891 (+6 ms)Cleaning up detritus from prior splits at 1675983346891Running coprocessor post-open hooks at 1675983346893 (+2 ms)Region opened successfully at 1675983346893 2023-02-09 22:55:46,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224., pid=133, masterSystemTime=1675983346881 2023-02-09 22:55:46,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. 2023-02-09 22:55:46,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. 2023-02-09 22:55:46,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:46,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6eacad5bba92e31c8debd2219e721254, NAME => 'testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.', STARTKEY => '', ENDKEY => '71'} 2023-02-09 22:55:46,895 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=9813e50bcd0989a84947ee998d75c224, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:46,895 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983346895"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983346895"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983346895"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983346895"}]},"ts":"1675983346895"} 2023-02-09 22:55:46,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:46,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,897 INFO [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,898 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1 2023-02-09 22:55:46,898 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1 2023-02-09 22:55:46,898 INFO [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6eacad5bba92e31c8debd2219e721254 columnFamilyName cf1 2023-02-09 22:55:46,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-09 22:55:46,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 9813e50bcd0989a84947ee998d75c224, server=jenkins-hbase9.apache.org,40413,1675983241505 in 167 msec 2023-02-09 22:55:46,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9813e50bcd0989a84947ee998d75c224, ASSIGN in 322 msec 2023-02-09 22:55:46,903 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-3c5a1c55c0654a33931974cef314bc88 2023-02-09 22:55:46,906 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4ae0acc237224157aa55e5f9b30fdada 2023-02-09 22:55:46,908 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-55e40d05dda14c20a40f5209a3006909 2023-02-09 22:55:46,911 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-5c8e0dea26f143b789f63823450e6265 2023-02-09 22:55:46,913 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-aaefeb9a6154455bb1a33da5617552e9 2023-02-09 22:55:46,916 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-d6416cc775bd443fa661a73dd36d5a39 2023-02-09 22:55:46,918 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f44b9828c98c4f72ad2986fed4ee3134 2023-02-09 22:55:46,918 INFO [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(310): Store=6eacad5bba92e31c8debd2219e721254/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:46,918 INFO [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,919 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2 2023-02-09 22:55:46,919 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2 2023-02-09 22:55:46,919 INFO [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6eacad5bba92e31c8debd2219e721254 columnFamilyName cf2 2023-02-09 22:55:46,923 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-26b984b169d248d984ee5f9123374416 2023-02-09 22:55:46,925 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4605288aa5b84830a2a2948ae89a7be0 2023-02-09 22:55:46,927 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-47cdd6751dcd491c91910a7b0ddb8fcb 2023-02-09 22:55:46,929 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-71d18d647b54448d83251165f2580ca8 2023-02-09 22:55:46,931 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-9ace2b3927fa40e0844f5ae99b3c53e9 2023-02-09 22:55:46,933 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-a1698bb0cedf4eb2a4ed9242b4fce527 2023-02-09 22:55:46,935 DEBUG [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f3f5e7d5ca3e4c81ae5d3f489cbcc75e 2023-02-09 22:55:46,935 INFO [StoreOpener-6eacad5bba92e31c8debd2219e721254-1] regionserver.HStore(310): Store=6eacad5bba92e31c8debd2219e721254/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:46,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:46,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:46,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 6eacad5bba92e31c8debd2219e721254; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75161436, jitterRate=0.11999267339706421}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:46,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 6eacad5bba92e31c8debd2219e721254: Running coprocessor pre-open hook at 1675983346896Writing region info on filesystem at 1675983346896Initializing all the Stores at 1675983346896Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983346896Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983346897 (+1 ms)Cleaning up temporary data from old regions at 1675983346937 (+40 ms)Cleaning up detritus from prior splits at 1675983346937Running coprocessor post-open hooks at 1675983346938 (+1 ms)Region opened successfully at 1675983346938 2023-02-09 22:55:46,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254., pid=134, masterSystemTime=1675983346881 2023-02-09 22:55:46,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:46,940 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:46,940 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 22:55:46,940 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1912): 6eacad5bba92e31c8debd2219e721254/cf1 is initiating minor compaction (all files) 2023-02-09 22:55:46,941 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6eacad5bba92e31c8debd2219e721254/cf1 in testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:46,941 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f44b9828c98c4f72ad2986fed4ee3134, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-d6416cc775bd443fa661a73dd36d5a39, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4ae0acc237224157aa55e5f9b30fdada, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-aaefeb9a6154455bb1a33da5617552e9, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-5c8e0dea26f143b789f63823450e6265, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-55e40d05dda14c20a40f5209a3006909, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-3c5a1c55c0654a33931974cef314bc88] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/.tmp, totalSize=34.9 K 2023-02-09 22:55:46,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:55:46,941 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:46,941 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f44b9828c98c4f72ad2986fed4ee3134, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983344431 2023-02-09 22:55:46,941 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 22:55:46,941 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1912): 6eacad5bba92e31c8debd2219e721254/cf2 is initiating minor compaction (all files) 2023-02-09 22:55:46,941 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6eacad5bba92e31c8debd2219e721254/cf2 in testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:46,941 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-d6416cc775bd443fa661a73dd36d5a39, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983344598 2023-02-09 22:55:46,941 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-26b984b169d248d984ee5f9123374416, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-9ace2b3927fa40e0844f5ae99b3c53e9, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f3f5e7d5ca3e4c81ae5d3f489cbcc75e, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-47cdd6751dcd491c91910a7b0ddb8fcb, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-71d18d647b54448d83251165f2580ca8, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4605288aa5b84830a2a2948ae89a7be0, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-a1698bb0cedf4eb2a4ed9242b4fce527] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/.tmp, totalSize=34.9 K 2023-02-09 22:55:46,942 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4ae0acc237224157aa55e5f9b30fdada, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983345566 2023-02-09 22:55:46,942 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-26b984b169d248d984ee5f9123374416, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983344431 2023-02-09 22:55:46,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:46,942 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-aaefeb9a6154455bb1a33da5617552e9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983345737 2023-02-09 22:55:46,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:46,942 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-9ace2b3927fa40e0844f5ae99b3c53e9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983344598 2023-02-09 22:55:46,942 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6eacad5bba92e31c8debd2219e721254, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:46,942 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-5c8e0dea26f143b789f63823450e6265, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983345911 2023-02-09 22:55:46,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983346942"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983346942"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983346942"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983346942"}]},"ts":"1675983346942"} 2023-02-09 22:55:46,943 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f3f5e7d5ca3e4c81ae5d3f489cbcc75e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983345566 2023-02-09 22:55:46,943 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-55e40d05dda14c20a40f5209a3006909, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983346093 2023-02-09 22:55:46,943 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-47cdd6751dcd491c91910a7b0ddb8fcb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983345737 2023-02-09 22:55:46,943 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-3c5a1c55c0654a33931974cef314bc88, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983346263 2023-02-09 22:55:46,943 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-71d18d647b54448d83251165f2580ca8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983345911 2023-02-09 22:55:46,944 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4605288aa5b84830a2a2948ae89a7be0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983346093 2023-02-09 22:55:46,944 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-a1698bb0cedf4eb2a4ed9242b4fce527, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983346263 2023-02-09 22:55:46,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-09 22:55:46,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 6eacad5bba92e31c8debd2219e721254, server=jenkins-hbase9.apache.org,40413,1675983241505 in 211 msec 2023-02-09 22:55:46,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-09 22:55:46,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6eacad5bba92e31c8debd2219e721254, ASSIGN in 370 msec 2023-02-09 22:55:46,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3c1463347f30b7be526325deae6a0aa0, daughterA=6eacad5bba92e31c8debd2219e721254, daughterB=9813e50bcd0989a84947ee998d75c224 in 679 msec 2023-02-09 22:55:46,970 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6eacad5bba92e31c8debd2219e721254#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:46,970 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] throttle.PressureAwareThroughputController(145): 6eacad5bba92e31c8debd2219e721254#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:46,980 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/.tmp/cf2/7637c2b4adef4eec8719a8d262cb6293 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/7637c2b4adef4eec8719a8d262cb6293 2023-02-09 22:55:46,980 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/.tmp/cf1/58d343d270db479d93416456341f14c8 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/58d343d270db479d93416456341f14c8 2023-02-09 22:55:46,984 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6eacad5bba92e31c8debd2219e721254/cf2 of 6eacad5bba92e31c8debd2219e721254 into 7637c2b4adef4eec8719a8d262cb6293(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:46,984 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6eacad5bba92e31c8debd2219e721254/cf1 of 6eacad5bba92e31c8debd2219e721254 into 58d343d270db479d93416456341f14c8(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:46,984 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6eacad5bba92e31c8debd2219e721254: 2023-02-09 22:55:46,984 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6eacad5bba92e31c8debd2219e721254: 2023-02-09 22:55:46,984 INFO [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254., storeName=6eacad5bba92e31c8debd2219e721254/cf2, priority=9, startTime=1675983346941; duration=0sec 2023-02-09 22:55:46,984 INFO [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254., storeName=6eacad5bba92e31c8debd2219e721254/cf1, priority=9, startTime=1675983346939; duration=0sec 2023-02-09 22:55:46,985 DEBUG [RS:2;jenkins-hbase9:40413-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:46,986 DEBUG [RS:2;jenkins-hbase9:40413-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:47,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40413] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.2.10:47988 deadline: 1675983407020, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0. is not online on jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:47,080 ERROR [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid b11f84d6-5ece-443b-9b52-3e4a014154a8) service to localhost/127.0.0.1:41659 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:55:47,131 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-09 22:55:47,134 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:47,135 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,136 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 22:55:47,143 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983347143"}]},"ts":"1675983347143"} 2023-02-09 22:55:47,144 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-09 22:55:47,146 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-09 22:55:47,147 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6eacad5bba92e31c8debd2219e721254, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9813e50bcd0989a84947ee998d75c224, UNASSIGN}] 2023-02-09 22:55:47,147 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9813e50bcd0989a84947ee998d75c224, UNASSIGN 2023-02-09 22:55:47,151 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=9813e50bcd0989a84947ee998d75c224, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:47,151 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983347151"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983347151"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983347151"}]},"ts":"1675983347151"} 2023-02-09 22:55:47,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 9813e50bcd0989a84947ee998d75c224, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:47,155 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6eacad5bba92e31c8debd2219e721254, UNASSIGN 2023-02-09 22:55:47,156 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6eacad5bba92e31c8debd2219e721254, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:47,156 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983347156"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983347156"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983347156"}]},"ts":"1675983347156"} 2023-02-09 22:55:47,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 6eacad5bba92e31c8debd2219e721254, server=jenkins-hbase9.apache.org,40413,1675983241505}] 2023-02-09 22:55:47,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 22:55:47,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:47,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 9813e50bcd0989a84947ee998d75c224, disabling compactions & flushes 2023-02-09 22:55:47,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. 2023-02-09 22:55:47,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. 2023-02-09 22:55:47,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. after waiting 0 ms 2023-02-09 22:55:47,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. 2023-02-09 22:55:47,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-09 22:55:47,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:47,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224. 2023-02-09 22:55:47,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 9813e50bcd0989a84947ee998d75c224: Waiting for close lock at 1675983347305Running coprocessor pre-close hooks at 1675983347305Disabling compacts and flushes for region at 1675983347305Disabling writes for close at 1675983347306 (+1 ms)Writing region close event to WAL at 1675983347306Running coprocessor post-close hooks at 1675983347309 (+3 ms)Closed at 1675983347309 2023-02-09 22:55:47,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:47,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:47,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 6eacad5bba92e31c8debd2219e721254, disabling compactions & flushes 2023-02-09 22:55:47,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:47,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:47,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. after waiting 0 ms 2023-02-09 22:55:47,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:47,311 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=9813e50bcd0989a84947ee998d75c224, regionState=CLOSED 2023-02-09 22:55:47,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983347311"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983347311"}]},"ts":"1675983347311"} 2023-02-09 22:55:47,311 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f44b9828c98c4f72ad2986fed4ee3134, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-d6416cc775bd443fa661a73dd36d5a39, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4ae0acc237224157aa55e5f9b30fdada, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-aaefeb9a6154455bb1a33da5617552e9, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-5c8e0dea26f143b789f63823450e6265, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-55e40d05dda14c20a40f5209a3006909, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-3c5a1c55c0654a33931974cef314bc88] to archive 2023-02-09 22:55:47,312 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:47,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-09 22:55:47,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 9813e50bcd0989a84947ee998d75c224, server=jenkins-hbase9.apache.org,40413,1675983241505 in 160 msec 2023-02-09 22:55:47,314 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f44b9828c98c4f72ad2986fed4ee3134 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f44b9828c98c4f72ad2986fed4ee3134 2023-02-09 22:55:47,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9813e50bcd0989a84947ee998d75c224, UNASSIGN in 168 msec 2023-02-09 22:55:47,315 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-d6416cc775bd443fa661a73dd36d5a39 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-d6416cc775bd443fa661a73dd36d5a39 2023-02-09 22:55:47,316 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4ae0acc237224157aa55e5f9b30fdada to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4ae0acc237224157aa55e5f9b30fdada 2023-02-09 22:55:47,317 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-aaefeb9a6154455bb1a33da5617552e9 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-aaefeb9a6154455bb1a33da5617552e9 2023-02-09 22:55:47,318 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-5c8e0dea26f143b789f63823450e6265 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-5c8e0dea26f143b789f63823450e6265 2023-02-09 22:55:47,319 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-55e40d05dda14c20a40f5209a3006909 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-55e40d05dda14c20a40f5209a3006909 2023-02-09 22:55:47,320 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-3c5a1c55c0654a33931974cef314bc88 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-3c5a1c55c0654a33931974cef314bc88 2023-02-09 22:55:47,321 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-26b984b169d248d984ee5f9123374416, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-9ace2b3927fa40e0844f5ae99b3c53e9, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f3f5e7d5ca3e4c81ae5d3f489cbcc75e, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-47cdd6751dcd491c91910a7b0ddb8fcb, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-71d18d647b54448d83251165f2580ca8, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4605288aa5b84830a2a2948ae89a7be0, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-a1698bb0cedf4eb2a4ed9242b4fce527] to archive 2023-02-09 22:55:47,321 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:47,322 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-26b984b169d248d984ee5f9123374416 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-26b984b169d248d984ee5f9123374416 2023-02-09 22:55:47,323 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-9ace2b3927fa40e0844f5ae99b3c53e9 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-9ace2b3927fa40e0844f5ae99b3c53e9 2023-02-09 22:55:47,325 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f3f5e7d5ca3e4c81ae5d3f489cbcc75e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-f3f5e7d5ca3e4c81ae5d3f489cbcc75e 2023-02-09 22:55:47,325 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-47cdd6751dcd491c91910a7b0ddb8fcb to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-47cdd6751dcd491c91910a7b0ddb8fcb 2023-02-09 22:55:47,326 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-71d18d647b54448d83251165f2580ca8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-71d18d647b54448d83251165f2580ca8 2023-02-09 22:55:47,327 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4605288aa5b84830a2a2948ae89a7be0 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-4605288aa5b84830a2a2948ae89a7be0 2023-02-09 22:55:47,328 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-a1698bb0cedf4eb2a4ed9242b4fce527 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/testSplitTableRegionEmptyDaughter=3c1463347f30b7be526325deae6a0aa0-a1698bb0cedf4eb2a4ed9242b4fce527 2023-02-09 22:55:47,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:55:47,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:47,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254. 2023-02-09 22:55:47,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 6eacad5bba92e31c8debd2219e721254: Waiting for close lock at 1675983347311Running coprocessor pre-close hooks at 1675983347311Disabling compacts and flushes for region at 1675983347311Disabling writes for close at 1675983347311Writing region close event to WAL at 1675983347328 (+17 ms)Running coprocessor post-close hooks at 1675983347331 (+3 ms)Closed at 1675983347331 2023-02-09 22:55:47,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:47,332 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6eacad5bba92e31c8debd2219e721254, regionState=CLOSED 2023-02-09 22:55:47,332 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983347332"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983347332"}]},"ts":"1675983347332"} 2023-02-09 22:55:47,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-09 22:55:47,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 6eacad5bba92e31c8debd2219e721254, server=jenkins-hbase9.apache.org,40413,1675983241505 in 176 msec 2023-02-09 22:55:47,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-09 22:55:47,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6eacad5bba92e31c8debd2219e721254, UNASSIGN in 188 msec 2023-02-09 22:55:47,336 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983347336"}]},"ts":"1675983347336"} 2023-02-09 22:55:47,337 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-09 22:55:47,339 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-09 22:55:47,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 203 msec 2023-02-09 22:55:47,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 22:55:47,440 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-09 22:55:47,440 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 22:55:47,447 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:47,447 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:47,447 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:47,448 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/recovered.edits] 2023-02-09 22:55:47,448 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/recovered.edits] 2023-02-09 22:55:47,449 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/recovered.edits] 2023-02-09 22:55:47,451 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/58d343d270db479d93416456341f14c8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf1/58d343d270db479d93416456341f14c8 2023-02-09 22:55:47,451 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/3c5a1c55c0654a33931974cef314bc88 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/3c5a1c55c0654a33931974cef314bc88 2023-02-09 22:55:47,452 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/4ae0acc237224157aa55e5f9b30fdada to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/4ae0acc237224157aa55e5f9b30fdada 2023-02-09 22:55:47,453 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/7637c2b4adef4eec8719a8d262cb6293 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/cf2/7637c2b4adef4eec8719a8d262cb6293 2023-02-09 22:55:47,453 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/recovered.edits/88.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224/recovered.edits/88.seqid 2023-02-09 22:55:47,453 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/55e40d05dda14c20a40f5209a3006909 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/55e40d05dda14c20a40f5209a3006909 2023-02-09 22:55:47,454 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/9813e50bcd0989a84947ee998d75c224 2023-02-09 22:55:47,455 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/5c8e0dea26f143b789f63823450e6265 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/5c8e0dea26f143b789f63823450e6265 2023-02-09 22:55:47,456 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/aaefeb9a6154455bb1a33da5617552e9 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/aaefeb9a6154455bb1a33da5617552e9 2023-02-09 22:55:47,456 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254/recovered.edits/90.seqid 2023-02-09 22:55:47,456 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/6eacad5bba92e31c8debd2219e721254 2023-02-09 22:55:47,457 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/d6416cc775bd443fa661a73dd36d5a39 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/d6416cc775bd443fa661a73dd36d5a39 2023-02-09 22:55:47,458 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/f44b9828c98c4f72ad2986fed4ee3134 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf1/f44b9828c98c4f72ad2986fed4ee3134 2023-02-09 22:55:47,460 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/26b984b169d248d984ee5f9123374416 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/26b984b169d248d984ee5f9123374416 2023-02-09 22:55:47,461 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/4605288aa5b84830a2a2948ae89a7be0 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/4605288aa5b84830a2a2948ae89a7be0 2023-02-09 22:55:47,462 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/47cdd6751dcd491c91910a7b0ddb8fcb to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/47cdd6751dcd491c91910a7b0ddb8fcb 2023-02-09 22:55:47,463 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/71d18d647b54448d83251165f2580ca8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/71d18d647b54448d83251165f2580ca8 2023-02-09 22:55:47,464 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/9ace2b3927fa40e0844f5ae99b3c53e9 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/9ace2b3927fa40e0844f5ae99b3c53e9 2023-02-09 22:55:47,465 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/a1698bb0cedf4eb2a4ed9242b4fce527 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/a1698bb0cedf4eb2a4ed9242b4fce527 2023-02-09 22:55:47,466 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/cf2/f3f5e7d5ca3e4c81ae5d3f489cbcc75e 2023-02-09 22:55:47,468 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/recovered.edits/85.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0/recovered.edits/85.seqid 2023-02-09 22:55:47,468 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegionEmptyDaughter/3c1463347f30b7be526325deae6a0aa0 2023-02-09 22:55:47,468 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 22:55:47,470 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,472 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-09 22:55:47,473 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-09 22:55:47,474 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,474 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-09 22:55:47,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983347474"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:47,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983347474"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:47,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983347474"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:47,475 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:55:47,476 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3c1463347f30b7be526325deae6a0aa0, NAME => 'testSplitTableRegionEmptyDaughter,,1675983343927.3c1463347f30b7be526325deae6a0aa0.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6eacad5bba92e31c8debd2219e721254, NAME => 'testSplitTableRegionEmptyDaughter,,1675983346269.6eacad5bba92e31c8debd2219e721254.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 9813e50bcd0989a84947ee998d75c224, NAME => 'testSplitTableRegionEmptyDaughter,71,1675983346269.9813e50bcd0989a84947ee998d75c224.', STARTKEY => '71', ENDKEY => ''}] 2023-02-09 22:55:47,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-09 22:55:47,476 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983347476"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:47,477 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-09 22:55:47,478 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 22:55:47,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 38 msec 2023-02-09 22:55:47,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 22:55:47,544 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-09 22:55:47,558 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=373 (was 376), OpenFileDescriptor=836 (was 840), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=126 (was 120) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=10637 (was 10640) 2023-02-09 22:55:47,571 INFO [Listener at localhost/44743] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=373, OpenFileDescriptor=836, MaxFileDescriptor=60000, SystemLoadAverage=126, ProcessCount=168, AvailableMemoryMB=10636 2023-02-09 22:55:47,571 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:47,574 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 22:55:47,575 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(492): Client=jenkins//172.31.2.10 set balanceSwitch=false 2023-02-09 22:55:47,576 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-09 22:55:47,578 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:47,578 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:47,580 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:47,580 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158 empty. 2023-02-09 22:55:47,581 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:47,581 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 22:55:48,797 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:48,799 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 056dfa2d1fc16a781dc92e935f5f2158, NAME => 'testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp 2023-02-09 22:55:49,210 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:49,210 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 056dfa2d1fc16a781dc92e935f5f2158, disabling compactions & flushes 2023-02-09 22:55:49,210 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,210 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,210 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. after waiting 0 ms 2023-02-09 22:55:49,210 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,210 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,210 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 056dfa2d1fc16a781dc92e935f5f2158: Waiting for close lock at 1675983349210Disabling compacts and flushes for region at 1675983349210Disabling writes for close at 1675983349210Writing region close event to WAL at 1675983349210Closed at 1675983349210 2023-02-09 22:55:49,212 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:49,213 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983349213"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983349213"}]},"ts":"1675983349213"} 2023-02-09 22:55:49,214 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:55:49,215 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:49,215 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983349215"}]},"ts":"1675983349215"} 2023-02-09 22:55:49,216 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-09 22:55:49,219 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase9.apache.org=0} racks are {/default-rack=0} 2023-02-09 22:55:49,219 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 22:55:49,220 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 22:55:49,220 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 22:55:49,220 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 22:55:49,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=056dfa2d1fc16a781dc92e935f5f2158, ASSIGN}] 2023-02-09 22:55:49,221 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=056dfa2d1fc16a781dc92e935f5f2158, ASSIGN 2023-02-09 22:55:49,221 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=056dfa2d1fc16a781dc92e935f5f2158, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=false 2023-02-09 22:55:49,371 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 22:55:49,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=056dfa2d1fc16a781dc92e935f5f2158, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,373 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983349372"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983349372"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983349372"}]},"ts":"1675983349372"} 2023-02-09 22:55:49,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 056dfa2d1fc16a781dc92e935f5f2158, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:55:49,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 056dfa2d1fc16a781dc92e935f5f2158, NAME => 'testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:49,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:49,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:49,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,532 INFO [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,534 DEBUG [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1 2023-02-09 22:55:49,534 DEBUG [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1 2023-02-09 22:55:49,534 INFO [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 056dfa2d1fc16a781dc92e935f5f2158 columnFamilyName cf1 2023-02-09 22:55:49,534 INFO [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] regionserver.HStore(310): Store=056dfa2d1fc16a781dc92e935f5f2158/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:49,534 INFO [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,535 DEBUG [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2 2023-02-09 22:55:49,535 DEBUG [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2 2023-02-09 22:55:49,536 INFO [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 056dfa2d1fc16a781dc92e935f5f2158 columnFamilyName cf2 2023-02-09 22:55:49,536 INFO [StoreOpener-056dfa2d1fc16a781dc92e935f5f2158-1] regionserver.HStore(310): Store=056dfa2d1fc16a781dc92e935f5f2158/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:49,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:49,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:49,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:49,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 056dfa2d1fc16a781dc92e935f5f2158; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75190155, jitterRate=0.12042061984539032}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:49,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 056dfa2d1fc16a781dc92e935f5f2158: Running coprocessor pre-open hook at 1675983349532Writing region info on filesystem at 1675983349532Initializing all the Stores at 1675983349532Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983349532Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983349532Cleaning up temporary data from old regions at 1675983349537 (+5 ms)Cleaning up detritus from prior splits at 1675983349537Running coprocessor post-open hooks at 1675983349541 (+4 ms)Region opened successfully at 1675983349541 2023-02-09 22:55:49,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158., pid=143, masterSystemTime=1675983349525 2023-02-09 22:55:49,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,543 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=056dfa2d1fc16a781dc92e935f5f2158, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,543 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983349543"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983349543"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983349543"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983349543"}]},"ts":"1675983349543"} 2023-02-09 22:55:49,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-09 22:55:49,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 056dfa2d1fc16a781dc92e935f5f2158, server=jenkins-hbase9.apache.org,44627,1675983241473 in 170 msec 2023-02-09 22:55:49,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-09 22:55:49,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=056dfa2d1fc16a781dc92e935f5f2158, ASSIGN in 325 msec 2023-02-09 22:55:49,546 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:49,546 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983349546"}]},"ts":"1675983349546"} 2023-02-09 22:55:49,547 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-09 22:55:49,549 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:49,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 1.9740 sec 2023-02-09 22:55:49,582 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:49,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 22:55:49,584 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 22:55:49,584 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,584 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 22:55:49,584 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 22:55:49,584 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,584 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,586 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,587 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,587 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,587 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,588 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,589 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,589 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,589 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,589 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,589 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:49,589 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:49,589 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,589 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. started... 2023-02-09 22:55:49,590 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 056dfa2d1fc16a781dc92e935f5f2158 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 22:55:49,590 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,590 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,590 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,590 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,590 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,590 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,598 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/583cf4dfe12e4ef38cdeba91aa2026ad 2023-02-09 22:55:49,612 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e 2023-02-09 22:55:49,616 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/583cf4dfe12e4ef38cdeba91aa2026ad as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad 2023-02-09 22:55:49,618 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:49,619 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e 2023-02-09 22:55:49,622 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:49,622 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 32ms, sequenceid=6, compaction requested=false 2023-02-09 22:55:49,623 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-09 22:55:49,623 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 056dfa2d1fc16a781dc92e935f5f2158: 2023-02-09 22:55:49,623 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:49,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:49,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,623 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,625 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,625 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,625 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,625 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 22:55:49,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fb615da[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:49,627 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 22:55:49,627 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,628 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,628 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,629 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:49,628 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:49,634 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,634 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:55:49,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 22:55:49,636 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 22:55:49,637 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:49,637 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:49,637 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 22:55:49,637 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:49,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,638 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,638 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,639 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,640 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,641 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,641 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,641 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,644 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 22:55:49,644 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,644 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:49,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:49,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,644 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,645 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,646 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,745 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:49,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:49,755 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:49,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:49,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 22:55:49,756 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 22:55:49,756 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,756 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 22:55:49,756 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 22:55:49,757 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,757 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,758 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,758 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,758 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,758 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,758 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,758 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:49,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:49,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:49,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,760 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:49,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,761 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,761 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,761 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,761 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:49,761 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:49,761 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,761 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. started... 2023-02-09 22:55:49,761 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 056dfa2d1fc16a781dc92e935f5f2158 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:49,763 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,763 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,763 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,763 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,763 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,763 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,777 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/160321348e8446efa0687439e41937db 2023-02-09 22:55:49,788 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/d138fde8400f48fcb7098d33e5a40de7 2023-02-09 22:55:49,791 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/160321348e8446efa0687439e41937db as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db 2023-02-09 22:55:49,794 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:49,794 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/d138fde8400f48fcb7098d33e5a40de7 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7 2023-02-09 22:55:49,797 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 22:55:49,798 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 37ms, sequenceid=19, compaction requested=false 2023-02-09 22:55:49,798 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 056dfa2d1fc16a781dc92e935f5f2158: 2023-02-09 22:55:49,798 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,798 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:49,799 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:49,799 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,799 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,799 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,801 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,801 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 22:55:49,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@525d54ad[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:49,803 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 22:55:49,803 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,805 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:49,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,805 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:49,807 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,807 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,807 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,808 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,809 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,809 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,809 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,809 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,809 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,809 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:55:49,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 22:55:49,812 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 22:55:49,813 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:49,813 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:49,813 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 22:55:49,813 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:49,815 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,815 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,815 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,815 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,816 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,816 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,817 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,819 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,819 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,819 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,822 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 22:55:49,822 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,823 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,822 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,822 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,823 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,823 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,823 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,923 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:49,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:49,933 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:49,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:49,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 22:55:49,934 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 22:55:49,934 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,934 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 22:55:49,934 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 22:55:49,935 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,935 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,936 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,936 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,936 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,936 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,936 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,936 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:49,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:49,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:49,937 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:49,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:49,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:49,938 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:49,938 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,938 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. started... 2023-02-09 22:55:49,938 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 056dfa2d1fc16a781dc92e935f5f2158 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:49,940 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,947 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,947 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,947 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,947 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,947 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,950 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/c1eb04aad2fe40ce80a8d241f1bb635a 2023-02-09 22:55:49,959 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8 2023-02-09 22:55:49,962 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/c1eb04aad2fe40ce80a8d241f1bb635a as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a 2023-02-09 22:55:49,965 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:49,966 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8 2023-02-09 22:55:49,969 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 22:55:49,969 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 31ms, sequenceid=32, compaction requested=true 2023-02-09 22:55:49,969 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 056dfa2d1fc16a781dc92e935f5f2158: 2023-02-09 22:55:49,970 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:49,970 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:49,970 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:49,970 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,970 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:49,970 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:49,971 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,971 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,972 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,972 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 22:55:49,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dc7f182[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:49,973 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 22:55:49,973 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,975 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:49,975 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:49,976 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:49,977 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,977 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,977 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,977 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,977 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,978 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,977 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:49,978 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,978 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,978 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:55:49,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 22:55:49,980 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 22:55:49,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:49,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:49,980 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 22:55:49,980 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:49,982 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,982 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,982 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:49,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,983 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,983 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:49,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,984 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,985 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:49,985 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,987 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,987 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,987 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,987 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,988 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:49,988 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 22:55:49,988 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:49,987 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,988 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,988 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:49,988 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:49,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:49,988 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:49,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,988 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:49,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:49,988 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:49,988 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,989 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:49,989 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:49,989 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,989 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:49,989 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:49,989 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:49,989 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,089 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:50,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:50,099 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:50,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:50,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 22:55:50,100 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 22:55:50,101 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,101 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 22:55:50,101 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 22:55:50,101 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,101 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,103 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,103 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,103 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,103 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,103 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,103 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:50,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,105 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:50,105 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:50,105 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,106 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. started... 2023-02-09 22:55:50,106 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 056dfa2d1fc16a781dc92e935f5f2158 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:50,106 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,107 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,107 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,107 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,107 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,107 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,117 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7 2023-02-09 22:55:50,128 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/8061e3a06a35467fb171c2a3be3142ca 2023-02-09 22:55:50,132 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7 2023-02-09 22:55:50,135 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:50,135 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/8061e3a06a35467fb171c2a3be3142ca as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca 2023-02-09 22:55:50,138 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 22:55:50,139 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 33ms, sequenceid=45, compaction requested=true 2023-02-09 22:55:50,139 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 056dfa2d1fc16a781dc92e935f5f2158: 2023-02-09 22:55:50,139 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,139 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:50,139 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:50,139 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,139 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,139 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,141 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,141 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,141 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,141 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 22:55:50,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78321efc[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:50,143 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 22:55:50,143 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,145 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,145 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,145 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,146 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,146 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:50,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:50,146 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:50,147 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,147 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,147 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,149 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,149 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,149 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,149 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,149 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,149 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,149 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:55:50,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 22:55:50,153 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 22:55:50,153 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:50,153 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:50,153 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 22:55:50,153 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:50,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,157 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,157 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,158 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,159 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,160 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,160 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,170 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 22:55:50,170 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:50,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,170 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,170 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,171 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:50,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,171 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,171 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,171 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:50,171 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,172 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,172 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,172 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,172 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,172 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,172 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,272 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:50,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:50,283 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:50,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:50,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 22:55:50,284 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 22:55:50,284 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,284 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 22:55:50,284 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 22:55:50,285 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,285 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,286 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,286 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,286 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,286 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,286 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,287 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:50,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,288 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,288 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,288 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:50,288 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,288 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. started... 2023-02-09 22:55:50,288 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,289 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 056dfa2d1fc16a781dc92e935f5f2158 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:50,291 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,292 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,292 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,292 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,292 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,292 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,297 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/df5950e788884ce5a4b99c9d3f76fe4a 2023-02-09 22:55:50,306 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/7ab05cb802584a95b95cf6e2d7e13ccc 2023-02-09 22:55:50,309 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/df5950e788884ce5a4b99c9d3f76fe4a as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a 2023-02-09 22:55:50,312 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:50,312 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/7ab05cb802584a95b95cf6e2d7e13ccc as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc 2023-02-09 22:55:50,315 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 22:55:50,316 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 27ms, sequenceid=58, compaction requested=true 2023-02-09 22:55:50,316 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 056dfa2d1fc16a781dc92e935f5f2158: 2023-02-09 22:55:50,316 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,316 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:50,316 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:50,316 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,316 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,316 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,318 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,318 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,318 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,318 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 22:55:50,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27f5dce5[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:50,319 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 22:55:50,319 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,321 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:50,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:50,321 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:50,324 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,324 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,324 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,324 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,325 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,325 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,325 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,325 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,325 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,325 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:55:50,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 22:55:50,328 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 22:55:50,328 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:50,328 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:50,328 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 22:55:50,328 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:50,330 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,330 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,330 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,330 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,331 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,332 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,333 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,333 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,335 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,335 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,335 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,336 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,335 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,335 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,336 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 22:55:50,336 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:50,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:50,336 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,337 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,337 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,337 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,337 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,337 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,437 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:50,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:50,448 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:50,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:50,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 22:55:50,450 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 22:55:50,450 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,450 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 22:55:50,450 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 22:55:50,450 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,450 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,453 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,453 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,453 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,453 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,453 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,453 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 22:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,41885,1675983241428' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,40413,1675983241505' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:50,455 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:50,455 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,456 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. started... 2023-02-09 22:55:50,456 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 056dfa2d1fc16a781dc92e935f5f2158 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 22:55:50,456 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,457 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,457 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,458 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,458 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,458 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,463 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/c9ec8c71782f4ffb985913f14a80cf09 2023-02-09 22:55:50,473 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/3aca1d1ed93f440b84fb8fde326bb22a 2023-02-09 22:55:50,476 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/c9ec8c71782f4ffb985913f14a80cf09 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09 2023-02-09 22:55:50,479 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:50,479 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/3aca1d1ed93f440b84fb8fde326bb22a as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a 2023-02-09 22:55:50,482 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 22:55:50,483 INFO [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 27ms, sequenceid=71, compaction requested=true 2023-02-09 22:55:50,483 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 056dfa2d1fc16a781dc92e935f5f2158: 2023-02-09 22:55:50,483 DEBUG [rs(jenkins-hbase9.apache.org,44627,1675983241473)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,483 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:50,483 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:50,483 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,483 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 22:55:50,483 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 22:55:50,485 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,485 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,485 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,485 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,487 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,44627,1675983241473' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 22:55:50,487 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f351c8a[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:50,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 22:55:50,487 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,489 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,489 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,489 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,41885,1675983241428' in zk 2023-02-09 22:55:50,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,489 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:50,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,490 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,490 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 22:55:50,490 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,490 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,44627,1675983241473' in zk 2023-02-09 22:55:50,490 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 22:55:50,490 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase9.apache.org,40413,1675983241505' in zk 2023-02-09 22:55:50,498 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,498 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,498 DEBUG [member: 'jenkins-hbase9.apache.org,41885,1675983241428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,499 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,500 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,500 DEBUG [member: 'jenkins-hbase9.apache.org,40413,1675983241505' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,501 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 22:55:50,501 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,501 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,501 DEBUG [member: 'jenkins-hbase9.apache.org,44627,1675983241473' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 22:55:50,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase9.apache.org,44627,1675983241473': 2023-02-09 22:55:50,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,44627,1675983241473' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 22:55:50,504 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 22:55:50,504 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:50,504 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:50,504 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 22:55:50,504 INFO [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:50,505 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,505 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,505 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,505 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,505 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,505 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,506 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,506 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,506 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,507 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,508 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 22:55:50,508 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,509 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,511 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,511 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,511 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,512 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 22:55:50,512 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,512 DEBUG [(jenkins-hbase9.apache.org,44215,1675983240249)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 22:55:50,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,512 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,513 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:50,513 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:50,513 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 22:55:50,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,513 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 22:55:50,612 DEBUG [Listener at localhost/44743] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:50,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:50,619 DEBUG [Listener at localhost/44743] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,620 DEBUG [Listener at localhost/44743] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=056dfa2d1fc16a781dc92e935f5f2158, daughterA=2c5e9adabacb81a774c083e71e330be6, daughterB=3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:50,621 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=056dfa2d1fc16a781dc92e935f5f2158, daughterA=2c5e9adabacb81a774c083e71e330be6, daughterB=3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:50,621 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=056dfa2d1fc16a781dc92e935f5f2158, daughterA=2c5e9adabacb81a774c083e71e330be6, daughterB=3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:50,621 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=056dfa2d1fc16a781dc92e935f5f2158, daughterA=2c5e9adabacb81a774c083e71e330be6, daughterB=3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:50,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=056dfa2d1fc16a781dc92e935f5f2158, UNASSIGN}] 2023-02-09 22:55:50,623 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=056dfa2d1fc16a781dc92e935f5f2158, UNASSIGN 2023-02-09 22:55:50,624 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=056dfa2d1fc16a781dc92e935f5f2158, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:50,624 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983350624"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983350624"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983350624"}]},"ts":"1675983350624"} 2023-02-09 22:55:50,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 056dfa2d1fc16a781dc92e935f5f2158, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:55:50,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 056dfa2d1fc16a781dc92e935f5f2158, disabling compactions & flushes 2023-02-09 22:55:50,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. after waiting 0 ms 2023-02-09 22:55:50,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing 056dfa2d1fc16a781dc92e935f5f2158 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 22:55:50,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/81c3d48eae3942cd932c8e26f3753b66 2023-02-09 22:55:50,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/be09623031e240029e8654a30f782868 2023-02-09 22:55:50,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf1/81c3d48eae3942cd932c8e26f3753b66 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66 2023-02-09 22:55:50,807 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:50,807 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/.tmp/cf2/be09623031e240029e8654a30f782868 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868 2023-02-09 22:55:50,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 22:55:50,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 34ms, sequenceid=82, compaction requested=true 2023-02-09 22:55:50,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 22:55:50,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:50,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. 2023-02-09 22:55:50,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 056dfa2d1fc16a781dc92e935f5f2158: Waiting for close lock at 1675983350777Running coprocessor pre-close hooks at 1675983350777Disabling compacts and flushes for region at 1675983350777Disabling writes for close at 1675983350777Obtaining lock to block concurrent updates at 1675983350777Preparing flush snapshotting stores in 056dfa2d1fc16a781dc92e935f5f2158 at 1675983350777Finished memstore snapshotting testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675983350777Flushing stores of testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. at 1675983350778 (+1 ms)Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf1: creating writer at 1675983350779 (+1 ms)Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf1: appending metadata at 1675983350780 (+1 ms)Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf1: closing flushed file at 1675983350780Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf2: creating writer at 1675983350793 (+13 ms)Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf2: appending metadata at 1675983350794 (+1 ms)Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf2: closing flushed file at 1675983350794Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf1: reopening flushed file at 1675983350804 (+10 ms)Flushing 056dfa2d1fc16a781dc92e935f5f2158/cf2: reopening flushed file at 1675983350808 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 056dfa2d1fc16a781dc92e935f5f2158 in 34ms, sequenceid=82, compaction requested=true at 1675983350811 (+3 ms)Writing region close event to WAL at 1675983350813 (+2 ms)Running coprocessor post-close hooks at 1675983350816 (+3 ms)Closed at 1675983350816 2023-02-09 22:55:50,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,818 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=056dfa2d1fc16a781dc92e935f5f2158, regionState=CLOSED 2023-02-09 22:55:50,818 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983350818"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983350818"}]},"ts":"1675983350818"} 2023-02-09 22:55:50,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-09 22:55:50,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 056dfa2d1fc16a781dc92e935f5f2158, server=jenkins-hbase9.apache.org,44627,1675983241473 in 194 msec 2023-02-09 22:55:50,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-09 22:55:50,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=056dfa2d1fc16a781dc92e935f5f2158, UNASSIGN in 197 msec 2023-02-09 22:55:50,829 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=056dfa2d1fc16a781dc92e935f5f2158, threads=14 2023-02-09 22:55:50,829 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,829 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,829 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,830 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,830 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,830 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,830 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,831 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,831 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,831 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,831 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,835 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,835 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:50,835 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,006 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,008 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,015 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,017 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,042 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,051 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,052 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,052 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,056 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,056 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,454 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,454 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,455 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66 for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,458 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a for region: 056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:51,458 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 056dfa2d1fc16a781dc92e935f5f2158 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 22:55:51,501 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:55:51,503 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 22:55:51,504 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675983351504"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675983351504"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675983351504"}]},"ts":"1675983351504"} 2023-02-09 22:55:51,504 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983351504"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983351504"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983351504"}]},"ts":"1675983351504"} 2023-02-09 22:55:51,504 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983351504"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983351504"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983351504"}]},"ts":"1675983351504"} 2023-02-09 22:55:51,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2c5e9adabacb81a774c083e71e330be6, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3943b2bac2605401056b93f5cc63cbdf, ASSIGN}] 2023-02-09 22:55:51,507 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3943b2bac2605401056b93f5cc63cbdf, ASSIGN 2023-02-09 22:55:51,507 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=3943b2bac2605401056b93f5cc63cbdf, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=false 2023-02-09 22:55:51,508 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2c5e9adabacb81a774c083e71e330be6, ASSIGN 2023-02-09 22:55:51,508 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=2c5e9adabacb81a774c083e71e330be6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase9.apache.org,44627,1675983241473; forceNewPlan=false, retain=false 2023-02-09 22:55:51,658 INFO [jenkins-hbase9:44215] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 22:55:51,659 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=2c5e9adabacb81a774c083e71e330be6, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:51,659 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983351659"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983351659"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983351659"}]},"ts":"1675983351659"} 2023-02-09 22:55:51,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 2c5e9adabacb81a774c083e71e330be6, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:55:51,663 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=3943b2bac2605401056b93f5cc63cbdf, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:51,663 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983351663"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983351663"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983351663"}]},"ts":"1675983351663"} 2023-02-09 22:55:51,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 3943b2bac2605401056b93f5cc63cbdf, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:55:51,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:51,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2c5e9adabacb81a774c083e71e330be6, NAME => 'testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 22:55:51,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:51,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:51,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,816 INFO [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,816 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1 2023-02-09 22:55:51,817 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1 2023-02-09 22:55:51,817 INFO [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c5e9adabacb81a774c083e71e330be6 columnFamilyName cf1 2023-02-09 22:55:51,827 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db-bottom 2023-02-09 22:55:51,830 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7-bottom 2023-02-09 22:55:51,833 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad-bottom 2023-02-09 22:55:51,836 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66-bottom 2023-02-09 22:55:51,839 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a-bottom 2023-02-09 22:55:51,843 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09-bottom 2023-02-09 22:55:51,846 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a-bottom 2023-02-09 22:55:51,846 INFO [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(310): Store=2c5e9adabacb81a774c083e71e330be6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:51,846 INFO [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,846 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2 2023-02-09 22:55:51,847 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2 2023-02-09 22:55:51,847 INFO [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c5e9adabacb81a774c083e71e330be6 columnFamilyName cf2 2023-02-09 22:55:51,882 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a-bottom 2023-02-09 22:55:51,885 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc-bottom 2023-02-09 22:55:51,888 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca-bottom 2023-02-09 22:55:51,891 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8-bottom 2023-02-09 22:55:51,894 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e-bottom 2023-02-09 22:55:51,897 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868-bottom 2023-02-09 22:55:51,901 DEBUG [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7-bottom 2023-02-09 22:55:51,901 INFO [StoreOpener-2c5e9adabacb81a774c083e71e330be6-1] regionserver.HStore(310): Store=2c5e9adabacb81a774c083e71e330be6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:51,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:51,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:51,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 2c5e9adabacb81a774c083e71e330be6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71157434, jitterRate=0.060328394174575806}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:51,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 2c5e9adabacb81a774c083e71e330be6: Running coprocessor pre-open hook at 1675983351815Writing region info on filesystem at 1675983351815Initializing all the Stores at 1675983351815Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983351815Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983351816 (+1 ms)Cleaning up temporary data from old regions at 1675983351902 (+86 ms)Cleaning up detritus from prior splits at 1675983351903 (+1 ms)Running coprocessor post-open hooks at 1675983351904 (+1 ms)Region opened successfully at 1675983351904 2023-02-09 22:55:51,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6., pid=149, masterSystemTime=1675983351812 2023-02-09 22:55:51,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:51,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:55:51,906 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:51,906 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:51,906 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:51,906 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1912): 2c5e9adabacb81a774c083e71e330be6/cf1 is initiating minor compaction (all files) 2023-02-09 22:55:51,906 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:51,906 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2c5e9adabacb81a774c083e71e330be6/cf1 in testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:51,906 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1912): 2c5e9adabacb81a774c083e71e330be6/cf2 is initiating minor compaction (all files) 2023-02-09 22:55:51,906 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2c5e9adabacb81a774c083e71e330be6/cf2 in testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:51,906 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/.tmp, totalSize=34.9 K 2023-02-09 22:55:51,907 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868-bottom] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/.tmp, totalSize=34.9 K 2023-02-09 22:55:51,907 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983349579 2023-02-09 22:55:51,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:51,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:51,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:51,907 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675983349579 2023-02-09 22:55:51,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3943b2bac2605401056b93f5cc63cbdf, NAME => 'testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 22:55:51,907 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983349746 2023-02-09 22:55:51,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 22:55:51,908 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675983349746 2023-02-09 22:55:51,908 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983349924 2023-02-09 22:55:51,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,908 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=2c5e9adabacb81a774c083e71e330be6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:51,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:51,908 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983351908"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983351908"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983351908"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983351908"}]},"ts":"1675983351908"} 2023-02-09 22:55:51,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,908 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983350090 2023-02-09 22:55:51,908 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675983349924 2023-02-09 22:55:51,909 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983350273 2023-02-09 22:55:51,909 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675983350090 2023-02-09 22:55:51,909 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983350438 2023-02-09 22:55:51,909 INFO [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,910 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983350614 2023-02-09 22:55:51,910 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675983350273 2023-02-09 22:55:51,910 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1 2023-02-09 22:55:51,910 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1 2023-02-09 22:55:51,910 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675983350438 2023-02-09 22:55:51,911 INFO [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3943b2bac2605401056b93f5cc63cbdf columnFamilyName cf1 2023-02-09 22:55:51,911 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675983350614 2023-02-09 22:55:51,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-09 22:55:51,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 2c5e9adabacb81a774c083e71e330be6, server=jenkins-hbase9.apache.org,44627,1675983241473 in 249 msec 2023-02-09 22:55:51,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2c5e9adabacb81a774c083e71e330be6, ASSIGN in 406 msec 2023-02-09 22:55:51,920 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2c5e9adabacb81a774c083e71e330be6#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:51,922 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] throttle.PressureAwareThroughputController(145): 2c5e9adabacb81a774c083e71e330be6#cf2#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:51,924 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db-top 2023-02-09 22:55:51,926 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7-top 2023-02-09 22:55:51,928 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad-top 2023-02-09 22:55:51,934 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66-top 2023-02-09 22:55:51,937 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a-top 2023-02-09 22:55:51,940 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/.tmp/cf1/c27c9df897b04f9aa092779da7222ec4 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c27c9df897b04f9aa092779da7222ec4 2023-02-09 22:55:51,940 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09-top 2023-02-09 22:55:51,941 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/.tmp/cf2/f317a66e9d344a04bbf34f69d9d87fe7 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/f317a66e9d344a04bbf34f69d9d87fe7 2023-02-09 22:55:51,943 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a-top 2023-02-09 22:55:51,944 INFO [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(310): Store=3943b2bac2605401056b93f5cc63cbdf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:51,944 INFO [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,944 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2 2023-02-09 22:55:51,944 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2 2023-02-09 22:55:51,944 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2c5e9adabacb81a774c083e71e330be6/cf1 of 2c5e9adabacb81a774c083e71e330be6 into c27c9df897b04f9aa092779da7222ec4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:51,945 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2c5e9adabacb81a774c083e71e330be6: 2023-02-09 22:55:51,945 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6., storeName=2c5e9adabacb81a774c083e71e330be6/cf1, priority=9, startTime=1675983351905; duration=0sec 2023-02-09 22:55:51,945 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:51,945 INFO [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3943b2bac2605401056b93f5cc63cbdf columnFamilyName cf2 2023-02-09 22:55:51,946 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2c5e9adabacb81a774c083e71e330be6/cf2 of 2c5e9adabacb81a774c083e71e330be6 into f317a66e9d344a04bbf34f69d9d87fe7(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:51,946 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2c5e9adabacb81a774c083e71e330be6: 2023-02-09 22:55:51,946 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6., storeName=2c5e9adabacb81a774c083e71e330be6/cf2, priority=9, startTime=1675983351906; duration=0sec 2023-02-09 22:55:51,946 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:51,954 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a-top 2023-02-09 22:55:51,956 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc-top 2023-02-09 22:55:51,959 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca-top 2023-02-09 22:55:51,961 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:55:51,962 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8-top 2023-02-09 22:55:51,968 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e-top 2023-02-09 22:55:51,971 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868-top 2023-02-09 22:55:51,973 DEBUG [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(539): loaded hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7-top 2023-02-09 22:55:51,973 INFO [StoreOpener-3943b2bac2605401056b93f5cc63cbdf-1] regionserver.HStore(310): Store=3943b2bac2605401056b93f5cc63cbdf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:51,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:51,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:51,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 3943b2bac2605401056b93f5cc63cbdf; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59083189, jitterRate=-0.11959187686443329}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:51,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 3943b2bac2605401056b93f5cc63cbdf: Running coprocessor pre-open hook at 1675983351908Writing region info on filesystem at 1675983351908Initializing all the Stores at 1675983351909 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983351909Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983351909Cleaning up temporary data from old regions at 1675983351975 (+66 ms)Cleaning up detritus from prior splits at 1675983351976 (+1 ms)Running coprocessor post-open hooks at 1675983351978 (+2 ms)Region opened successfully at 1675983351978 2023-02-09 22:55:51,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf., pid=150, masterSystemTime=1675983351812 2023-02-09 22:55:51,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:51,980 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:51,980 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 22:55:51,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:55:51,980 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:51,980 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1912): 3943b2bac2605401056b93f5cc63cbdf/cf2 is initiating minor compaction (all files) 2023-02-09 22:55:51,980 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:51,980 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3943b2bac2605401056b93f5cc63cbdf/cf2 in testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:51,980 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1912): 3943b2bac2605401056b93f5cc63cbdf/cf1 is initiating minor compaction (all files) 2023-02-09 22:55:51,980 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3943b2bac2605401056b93f5cc63cbdf/cf1 in testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:51,980 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868-top] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/.tmp, totalSize=34.9 K 2023-02-09 22:55:51,980 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66-top] into tmpdir=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/.tmp, totalSize=34.9 K 2023-02-09 22:55:51,982 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983349579 2023-02-09 22:55:51,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:51,982 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675983349579 2023-02-09 22:55:51,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:51,982 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983349746 2023-02-09 22:55:51,983 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675983349746 2023-02-09 22:55:51,983 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983349924 2023-02-09 22:55:51,983 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675983349924 2023-02-09 22:55:51,984 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983350090 2023-02-09 22:55:51,984 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675983350090 2023-02-09 22:55:51,984 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675983350273 2023-02-09 22:55:51,984 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675983350273 2023-02-09 22:55:51,984 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting 3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675983350438 2023-02-09 22:55:51,985 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675983350438 2023-02-09 22:55:51,985 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] compactions.Compactor(207): Compacting be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983350614 2023-02-09 22:55:51,985 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] compactions.Compactor(207): Compacting 81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675983350614 2023-02-09 22:55:51,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=3943b2bac2605401056b93f5cc63cbdf, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:51,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983351989"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983351989"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983351989"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983351989"}]},"ts":"1675983351989"} 2023-02-09 22:55:51,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-09 22:55:51,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 3943b2bac2605401056b93f5cc63cbdf, server=jenkins-hbase9.apache.org,44627,1675983241473 in 327 msec 2023-02-09 22:55:51,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-09 22:55:51,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3943b2bac2605401056b93f5cc63cbdf, ASSIGN in 487 msec 2023-02-09 22:55:51,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=056dfa2d1fc16a781dc92e935f5f2158, daughterA=2c5e9adabacb81a774c083e71e330be6, daughterB=3943b2bac2605401056b93f5cc63cbdf in 1.3760 sec 2023-02-09 22:55:52,000 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] throttle.PressureAwareThroughputController(145): 3943b2bac2605401056b93f5cc63cbdf#cf2#compaction#144 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:52,000 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3943b2bac2605401056b93f5cc63cbdf#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:52,016 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/.tmp/cf2/bae58c2f5f2949a2900ebcf93d6dbd60 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/bae58c2f5f2949a2900ebcf93d6dbd60 2023-02-09 22:55:52,018 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/.tmp/cf1/f8dc100db5d243928c9b9fc071457c79 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/f8dc100db5d243928c9b9fc071457c79 2023-02-09 22:55:52,020 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3943b2bac2605401056b93f5cc63cbdf/cf2 of 3943b2bac2605401056b93f5cc63cbdf into bae58c2f5f2949a2900ebcf93d6dbd60(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:52,020 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3943b2bac2605401056b93f5cc63cbdf: 2023-02-09 22:55:52,020 INFO [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf., storeName=3943b2bac2605401056b93f5cc63cbdf/cf2, priority=9, startTime=1675983351980; duration=0sec 2023-02-09 22:55:52,020 DEBUG [RS:1;jenkins-hbase9:44627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:52,022 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3943b2bac2605401056b93f5cc63cbdf/cf1 of 3943b2bac2605401056b93f5cc63cbdf into f8dc100db5d243928c9b9fc071457c79(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:52,022 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3943b2bac2605401056b93f5cc63cbdf: 2023-02-09 22:55:52,022 INFO [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf., storeName=3943b2bac2605401056b93f5cc63cbdf/cf1, priority=9, startTime=1675983351979; duration=0sec 2023-02-09 22:55:52,022 DEBUG [RS:1;jenkins-hbase9:44627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:52,081 ERROR [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid b11f84d6-5ece-443b-9b52-3e4a014154a8) service to localhost/127.0.0.1:41659 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:55:52,082 ERROR [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid 4256d769-ee79-40f3-a328-ac20eb1446de) service to localhost/127.0.0.1:41659 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:55:52,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44627] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.2.10:44710 deadline: 1675983412125, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158. is not online on jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:52,235 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 22:55:52,240 INFO [Listener at localhost/44743] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 22:55:52,241 DEBUG [Listener at localhost/44743] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 22:55:52,248 WARN [Listener at localhost/44743] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 22:55:52,249 INFO [Listener at localhost/44743] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-09 22:55:52,250 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testSplitTableRegion 2023-02-09 22:55:52,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-09 22:55:52,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 22:55:52,252 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983352252"}]},"ts":"1675983352252"} 2023-02-09 22:55:52,253 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-09 22:55:52,255 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-09 22:55:52,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2c5e9adabacb81a774c083e71e330be6, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3943b2bac2605401056b93f5cc63cbdf, UNASSIGN}] 2023-02-09 22:55:52,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3943b2bac2605401056b93f5cc63cbdf, UNASSIGN 2023-02-09 22:55:52,256 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=3943b2bac2605401056b93f5cc63cbdf, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:52,256 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983352256"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983352256"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983352256"}]},"ts":"1675983352256"} 2023-02-09 22:55:52,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 3943b2bac2605401056b93f5cc63cbdf, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:55:52,259 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2c5e9adabacb81a774c083e71e330be6, UNASSIGN 2023-02-09 22:55:52,259 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=2c5e9adabacb81a774c083e71e330be6, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:52,259 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983352259"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983352259"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983352259"}]},"ts":"1675983352259"} 2023-02-09 22:55:52,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 2c5e9adabacb81a774c083e71e330be6, server=jenkins-hbase9.apache.org,44627,1675983241473}] 2023-02-09 22:55:52,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 22:55:52,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:52,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 2c5e9adabacb81a774c083e71e330be6, disabling compactions & flushes 2023-02-09 22:55:52,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:52,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:52,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. after waiting 0 ms 2023-02-09 22:55:52,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:52,410 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66-bottom] to archive 2023-02-09 22:55:52,410 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:52,411 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,412 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,413 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,414 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,415 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,416 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,417 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,418 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a-bottom, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868-bottom] to archive 2023-02-09 22:55:52,419 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:52,420 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,421 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,422 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,422 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,423 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,424 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,425 DEBUG [StoreCloser-testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:55:52,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:52,429 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6. 2023-02-09 22:55:52,429 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 2c5e9adabacb81a774c083e71e330be6: Waiting for close lock at 1675983352409Running coprocessor pre-close hooks at 1675983352409Disabling compacts and flushes for region at 1675983352409Disabling writes for close at 1675983352409Writing region close event to WAL at 1675983352425 (+16 ms)Running coprocessor post-close hooks at 1675983352428 (+3 ms)Closed at 1675983352429 (+1 ms) 2023-02-09 22:55:52,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:52,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:52,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 3943b2bac2605401056b93f5cc63cbdf, disabling compactions & flushes 2023-02-09 22:55:52,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:52,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:52,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. after waiting 0 ms 2023-02-09 22:55:52,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:52,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=2c5e9adabacb81a774c083e71e330be6, regionState=CLOSED 2023-02-09 22:55:52,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983352430"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983352430"}]},"ts":"1675983352430"} 2023-02-09 22:55:52,431 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66-top] to archive 2023-02-09 22:55:52,432 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:52,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-09 22:55:52,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 2c5e9adabacb81a774c083e71e330be6, server=jenkins-hbase9.apache.org,44627,1675983241473 in 172 msec 2023-02-09 22:55:52,434 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/583cf4dfe12e4ef38cdeba91aa2026ad.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2c5e9adabacb81a774c083e71e330be6, UNASSIGN in 178 msec 2023-02-09 22:55:52,436 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/160321348e8446efa0687439e41937db.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,437 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c1eb04aad2fe40ce80a8d241f1bb635a.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,438 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,438 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/df5950e788884ce5a4b99c9d3f76fe4a.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,439 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/c9ec8c71782f4ffb985913f14a80cf09.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,440 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/81c3d48eae3942cd932c8e26f3753b66.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,442 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a-top, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158->hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868-top] to archive 2023-02-09 22:55:52,442 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:52,443 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,444 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/d138fde8400f48fcb7098d33e5a40de7.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,445 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,446 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/8061e3a06a35467fb171c2a3be3142ca.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,447 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/7ab05cb802584a95b95cf6e2d7e13ccc.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,448 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/3aca1d1ed93f440b84fb8fde326bb22a.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,449 DEBUG [StoreCloser-testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/be09623031e240029e8654a30f782868.056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 22:55:52,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:52,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf. 2023-02-09 22:55:52,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 3943b2bac2605401056b93f5cc63cbdf: Waiting for close lock at 1675983352430Running coprocessor pre-close hooks at 1675983352430Disabling compacts and flushes for region at 1675983352430Disabling writes for close at 1675983352430Writing region close event to WAL at 1675983352449 (+19 ms)Running coprocessor post-close hooks at 1675983352452 (+3 ms)Closed at 1675983352453 (+1 ms) 2023-02-09 22:55:52,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:52,454 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=3943b2bac2605401056b93f5cc63cbdf, regionState=CLOSED 2023-02-09 22:55:52,455 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983352454"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983352454"}]},"ts":"1675983352454"} 2023-02-09 22:55:52,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-09 22:55:52,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 3943b2bac2605401056b93f5cc63cbdf, server=jenkins-hbase9.apache.org,44627,1675983241473 in 198 msec 2023-02-09 22:55:52,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-09 22:55:52,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3943b2bac2605401056b93f5cc63cbdf, UNASSIGN in 202 msec 2023-02-09 22:55:52,458 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983352458"}]},"ts":"1675983352458"} 2023-02-09 22:55:52,459 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-09 22:55:52,466 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-09 22:55:52,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 217 msec 2023-02-09 22:55:52,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 22:55:52,555 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-09 22:55:52,555 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testSplitTableRegion 2023-02-09 22:55:52,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 22:55:52,557 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 22:55:52,558 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 22:55:52,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 22:55:52,561 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,561 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:52,561 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:52,563 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/recovered.edits] 2023-02-09 22:55:52,563 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/recovered.edits] 2023-02-09 22:55:52,563 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2, FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/recovered.edits] 2023-02-09 22:55:52,566 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c27c9df897b04f9aa092779da7222ec4 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf1/c27c9df897b04f9aa092779da7222ec4 2023-02-09 22:55:52,566 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/f8dc100db5d243928c9b9fc071457c79 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf1/f8dc100db5d243928c9b9fc071457c79 2023-02-09 22:55:52,567 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/160321348e8446efa0687439e41937db 2023-02-09 22:55:52,568 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/f317a66e9d344a04bbf34f69d9d87fe7 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/cf2/f317a66e9d344a04bbf34f69d9d87fe7 2023-02-09 22:55:52,568 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/4d6ac6b6ea5943c7bc83a4f2c4246ab7 2023-02-09 22:55:52,568 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/bae58c2f5f2949a2900ebcf93d6dbd60 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/cf2/bae58c2f5f2949a2900ebcf93d6dbd60 2023-02-09 22:55:52,569 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/583cf4dfe12e4ef38cdeba91aa2026ad 2023-02-09 22:55:52,570 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6/recovered.edits/90.seqid 2023-02-09 22:55:52,571 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/81c3d48eae3942cd932c8e26f3753b66 2023-02-09 22:55:52,571 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/recovered.edits/90.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf/recovered.edits/90.seqid 2023-02-09 22:55:52,571 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/2c5e9adabacb81a774c083e71e330be6 2023-02-09 22:55:52,571 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/3943b2bac2605401056b93f5cc63cbdf 2023-02-09 22:55:52,572 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c1eb04aad2fe40ce80a8d241f1bb635a 2023-02-09 22:55:52,573 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/c9ec8c71782f4ffb985913f14a80cf09 2023-02-09 22:55:52,573 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf1/df5950e788884ce5a4b99c9d3f76fe4a 2023-02-09 22:55:52,575 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/3aca1d1ed93f440b84fb8fde326bb22a 2023-02-09 22:55:52,576 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/7ab05cb802584a95b95cf6e2d7e13ccc 2023-02-09 22:55:52,577 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8061e3a06a35467fb171c2a3be3142ca 2023-02-09 22:55:52,578 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/8b68b50d61f34e4eb6b9c1899e44e9d8 2023-02-09 22:55:52,579 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/b2c0f0ca0ab64b75bcd6dc006ff5ad1e 2023-02-09 22:55:52,580 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/be09623031e240029e8654a30f782868 2023-02-09 22:55:52,581 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7 to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/cf2/d138fde8400f48fcb7098d33e5a40de7 2023-02-09 22:55:52,583 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/recovered.edits/85.seqid to hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/archive/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158/recovered.edits/85.seqid 2023-02-09 22:55:52,583 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/.tmp/data/default/testSplitTableRegion/056dfa2d1fc16a781dc92e935f5f2158 2023-02-09 22:55:52,583 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 22:55:52,585 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 22:55:52,586 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-09 22:55:52,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-09 22:55:52,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 22:55:52,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-09 22:55:52,589 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983352588"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:52,589 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983352588"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:52,589 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983352588"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:52,590 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:55:52,590 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 056dfa2d1fc16a781dc92e935f5f2158, NAME => 'testSplitTableRegion,,1675983347575.056dfa2d1fc16a781dc92e935f5f2158.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2c5e9adabacb81a774c083e71e330be6, NAME => 'testSplitTableRegion,,1675983350619.2c5e9adabacb81a774c083e71e330be6.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 3943b2bac2605401056b93f5cc63cbdf, NAME => 'testSplitTableRegion,41,1675983350619.3943b2bac2605401056b93f5cc63cbdf.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 22:55:52,590 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-09 22:55:52,590 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983352590"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:52,591 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-09 22:55:52,594 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 22:55:52,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 39 msec 2023-02-09 22:55:52,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44215] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 22:55:52,659 INFO [Listener at localhost/44743] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-09 22:55:52,673 INFO [Listener at localhost/44743] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376 (was 373) - Thread LEAK? -, OpenFileDescriptor=856 (was 836) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=132 (was 126) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=10576 (was 10636) 2023-02-09 22:55:52,673 INFO [Listener at localhost/44743] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 22:55:52,673 INFO [Listener at localhost/44743] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:55:52,677 DEBUG [Listener at localhost/44743] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c8d5aec to 127.0.0.1:49544 2023-02-09 22:55:52,677 DEBUG [Listener at localhost/44743] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:52,678 DEBUG [Listener at localhost/44743] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 22:55:52,678 DEBUG [Listener at localhost/44743] util.JVMClusterUtil(257): Found active master hash=190209018, stopped=false 2023-02-09 22:55:52,678 INFO [Listener at localhost/44743] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:55:52,681 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 22:55:52,681 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 22:55:52,681 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 22:55:52,681 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 22:55:52,681 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:55:52,681 INFO [Listener at localhost/44743] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:55:52,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:55:52,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:55:52,681 DEBUG [Listener at localhost/44743] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x452b56e4 to 127.0.0.1:49544 2023-02-09 22:55:52,682 DEBUG [Listener at localhost/44743] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:52,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:55:52,682 INFO [Listener at localhost/44743] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,41885,1675983241428' ***** 2023-02-09 22:55:52,682 INFO [Listener at localhost/44743] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 22:55:52,682 INFO [Listener at localhost/44743] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,44627,1675983241473' ***** 2023-02-09 22:55:52,682 INFO [Listener at localhost/44743] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 22:55:52,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:55:52,682 INFO [Listener at localhost/44743] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,40413,1675983241505' ***** 2023-02-09 22:55:52,682 INFO [Listener at localhost/44743] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 22:55:52,682 INFO [RS:1;jenkins-hbase9:44627] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 22:55:52,682 INFO [RS:0;jenkins-hbase9:41885] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 22:55:52,683 INFO [RS:0;jenkins-hbase9:41885] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 22:55:52,683 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 22:55:52,683 INFO [RS:0;jenkins-hbase9:41885] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 22:55:52,683 INFO [RS:2;jenkins-hbase9:40413] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 22:55:52,683 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 22:55:52,683 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 22:55:52,683 INFO [RS:1;jenkins-hbase9:44627] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 22:55:52,683 INFO [RS:2;jenkins-hbase9:40413] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 22:55:52,683 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(3304): Received CLOSE for f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:55:52,683 INFO [RS:2;jenkins-hbase9:40413] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 22:55:52,683 INFO [RS:1;jenkins-hbase9:44627] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 22:55:52,684 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:52,684 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:52,683 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:52,684 DEBUG [RS:2;jenkins-hbase9:40413] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x736d636d to 127.0.0.1:49544 2023-02-09 22:55:52,684 DEBUG [RS:2;jenkins-hbase9:40413] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:52,684 DEBUG [RS:0;jenkins-hbase9:41885] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d2bef64 to 127.0.0.1:49544 2023-02-09 22:55:52,684 DEBUG [RS:1;jenkins-hbase9:44627] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e4776d0 to 127.0.0.1:49544 2023-02-09 22:55:52,684 DEBUG [RS:1;jenkins-hbase9:44627] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:52,684 DEBUG [RS:0;jenkins-hbase9:41885] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:52,684 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,40413,1675983241505; all regions closed. 2023-02-09 22:55:52,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing f0fb98f20238a9a9ac1a9a9d1be29f13, disabling compactions & flushes 2023-02-09 22:55:52,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:55:52,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:55:52,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. after waiting 0 ms 2023-02-09 22:55:52,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:55:52,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing f0fb98f20238a9a9ac1a9a9d1be29f13 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-09 22:55:52,684 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 22:55:52,684 INFO [RS:1;jenkins-hbase9:44627] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 22:55:52,841 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1479): Online Regions={f0fb98f20238a9a9ac1a9a9d1be29f13=hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13.} 2023-02-09 22:55:52,841 INFO [RS:1;jenkins-hbase9:44627] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 22:55:52,841 INFO [RS:1;jenkins-hbase9:44627] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 22:55:52,842 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 22:55:52,841 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:55:52,841 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:55:52,842 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 22:55:52,841 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:55:52,842 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-09 22:55:52,842 DEBUG [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1505): Waiting on f0fb98f20238a9a9ac1a9a9d1be29f13 2023-02-09 22:55:52,842 DEBUG [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 22:55:52,842 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 22:55:52,843 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 22:55:52,843 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 22:55:52,843 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 22:55:52,843 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 22:55:52,843 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-09 22:55:52,849 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 22:55:52,850 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 22:55:52,853 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 22:55:52,853 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 22:55:52,860 DEBUG [RS:2;jenkins-hbase9:40413] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs 2023-02-09 22:55:52,860 INFO [RS:2;jenkins-hbase9:40413] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C40413%2C1675983241505:(num 1675983243018) 2023-02-09 22:55:52,860 DEBUG [RS:2;jenkins-hbase9:40413] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:52,860 INFO [RS:2;jenkins-hbase9:40413] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:55:52,863 INFO [RS:2;jenkins-hbase9:40413] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 22:55:52,863 INFO [RS:2;jenkins-hbase9:40413] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 22:55:52,863 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 22:55:52,863 INFO [RS:2;jenkins-hbase9:40413] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 22:55:52,863 INFO [RS:2;jenkins-hbase9:40413] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 22:55:52,875 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/.tmp/info/098aa021fe374daca250890ca68436c0 2023-02-09 22:55:52,876 INFO [RS:2;jenkins-hbase9:40413] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:40413 2023-02-09 22:55:52,880 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/.tmp/info/65aa52c87f3e48d49c9f3957b8c5ccda 2023-02-09 22:55:52,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/.tmp/info/098aa021fe374daca250890ca68436c0 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/info/098aa021fe374daca250890ca68436c0 2023-02-09 22:55:52,883 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:52,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:52,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:55:52,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:55:52,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,40413,1675983241505 2023-02-09 22:55:52,884 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e6357f7 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1374) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:52,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:55:52,884 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a933d91 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:52,884 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:55:52,885 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65aa52c87f3e48d49c9f3957b8c5ccda 2023-02-09 22:55:52,885 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,40413,1675983241505] 2023-02-09 22:55:52,885 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,40413,1675983241505; numProcessing=1 2023-02-09 22:55:52,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/info/098aa021fe374daca250890ca68436c0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 22:55:52,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f0fb98f20238a9a9ac1a9a9d1be29f13 in 203ms, sequenceid=6, compaction requested=false 2023-02-09 22:55:52,888 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase9.apache.org,40413,1675983241505 already deleted, retry=false 2023-02-09 22:55:52,888 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,40413,1675983241505 expired; onlineServers=2 2023-02-09 22:55:52,903 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/.tmp/rep_barrier/f45374d9e26848a2b7cc61da66b3cf93 2023-02-09 22:55:52,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/namespace/f0fb98f20238a9a9ac1a9a9d1be29f13/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-09 22:55:52,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:52,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:55:52,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for f0fb98f20238a9a9ac1a9a9d1be29f13: Waiting for close lock at 1675983352684Running coprocessor pre-close hooks at 1675983352684Disabling compacts and flushes for region at 1675983352684Disabling writes for close at 1675983352684Obtaining lock to block concurrent updates at 1675983352685 (+1 ms)Preparing flush snapshotting stores in f0fb98f20238a9a9ac1a9a9d1be29f13 at 1675983352685Finished memstore snapshotting hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675983352685Flushing stores of hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. at 1675983352842 (+157 ms)Flushing f0fb98f20238a9a9ac1a9a9d1be29f13/info: creating writer at 1675983352842Flushing f0fb98f20238a9a9ac1a9a9d1be29f13/info: appending metadata at 1675983352851 (+9 ms)Flushing f0fb98f20238a9a9ac1a9a9d1be29f13/info: closing flushed file at 1675983352851Flushing f0fb98f20238a9a9ac1a9a9d1be29f13/info: reopening flushed file at 1675983352882 (+31 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f0fb98f20238a9a9ac1a9a9d1be29f13 in 203ms, sequenceid=6, compaction requested=false at 1675983352887 (+5 ms)Writing region close event to WAL at 1675983352889 (+2 ms)Running coprocessor post-close hooks at 1675983352904 (+15 ms)Closed at 1675983352905 (+1 ms) 2023-02-09 22:55:52,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675983243403.f0fb98f20238a9a9ac1a9a9d1be29f13. 2023-02-09 22:55:52,907 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f45374d9e26848a2b7cc61da66b3cf93 2023-02-09 22:55:52,914 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 22:55:52,914 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 22:55:52,919 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/.tmp/table/fea423ae21384f5094e22f3eb8bf493b 2023-02-09 22:55:52,922 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fea423ae21384f5094e22f3eb8bf493b 2023-02-09 22:55:52,923 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/.tmp/info/65aa52c87f3e48d49c9f3957b8c5ccda as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/info/65aa52c87f3e48d49c9f3957b8c5ccda 2023-02-09 22:55:52,926 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 65aa52c87f3e48d49c9f3957b8c5ccda 2023-02-09 22:55:52,926 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/info/65aa52c87f3e48d49c9f3957b8c5ccda, entries=59, sequenceid=210, filesize=11.1 K 2023-02-09 22:55:52,927 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/.tmp/rep_barrier/f45374d9e26848a2b7cc61da66b3cf93 as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/rep_barrier/f45374d9e26848a2b7cc61da66b3cf93 2023-02-09 22:55:52,930 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f45374d9e26848a2b7cc61da66b3cf93 2023-02-09 22:55:52,930 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/rep_barrier/f45374d9e26848a2b7cc61da66b3cf93, entries=24, sequenceid=210, filesize=7.5 K 2023-02-09 22:55:52,930 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/.tmp/table/fea423ae21384f5094e22f3eb8bf493b as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/table/fea423ae21384f5094e22f3eb8bf493b 2023-02-09 22:55:52,933 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fea423ae21384f5094e22f3eb8bf493b 2023-02-09 22:55:52,933 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/table/fea423ae21384f5094e22f3eb8bf493b, entries=36, sequenceid=210, filesize=8.0 K 2023-02-09 22:55:52,934 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 91ms, sequenceid=210, compaction requested=false 2023-02-09 22:55:52,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-09 22:55:52,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 22:55:52,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 22:55:52,942 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 22:55:52,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675983352842Running coprocessor pre-close hooks at 1675983352842Disabling compacts and flushes for region at 1675983352842Disabling writes for close at 1675983352843 (+1 ms)Obtaining lock to block concurrent updates at 1675983352843Preparing flush snapshotting stores in 1588230740 at 1675983352843Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1675983352843Flushing stores of hbase:meta,,1.1588230740 at 1675983352846 (+3 ms)Flushing 1588230740/info: creating writer at 1675983352846Flushing 1588230740/info: appending metadata at 1675983352854 (+8 ms)Flushing 1588230740/info: closing flushed file at 1675983352854Flushing 1588230740/rep_barrier: creating writer at 1675983352885 (+31 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675983352889 (+4 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675983352889Flushing 1588230740/table: creating writer at 1675983352908 (+19 ms)Flushing 1588230740/table: appending metadata at 1675983352910 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675983352910Flushing 1588230740/info: reopening flushed file at 1675983352924 (+14 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675983352927 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1675983352931 (+4 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 91ms, sequenceid=210, compaction requested=false at 1675983352934 (+3 ms)Writing region close event to WAL at 1675983352938 (+4 ms)Running coprocessor post-close hooks at 1675983352941 (+3 ms)Closed at 1675983352942 (+1 ms) 2023-02-09 22:55:52,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 22:55:52,987 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:52,987 INFO [RS:2;jenkins-hbase9:40413] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,40413,1675983241505; zookeeper connection closed. 2023-02-09 22:55:52,987 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@125c2d39 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:52,987 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:40413-0x1003b4b98c70003, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:52,987 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7c7d7949 rejected from java.util.concurrent.ThreadPoolExecutor@2a5cd6e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:52,987 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4a0aa464] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4a0aa464 2023-02-09 22:55:53,042 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,41885,1675983241428; all regions closed. 2023-02-09 22:55:53,043 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,44627,1675983241473; all regions closed. 2023-02-09 22:55:53,049 DEBUG [RS:0;jenkins-hbase9:41885] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs 2023-02-09 22:55:53,049 DEBUG [RS:1;jenkins-hbase9:44627] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs 2023-02-09 22:55:53,049 INFO [RS:1;jenkins-hbase9:44627] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C44627%2C1675983241473.meta:.meta(num 1675983243131) 2023-02-09 22:55:53,049 INFO [RS:0;jenkins-hbase9:41885] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C41885%2C1675983241428:(num 1675983243019) 2023-02-09 22:55:53,049 DEBUG [RS:0;jenkins-hbase9:41885] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:53,049 INFO [RS:0;jenkins-hbase9:41885] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:55:53,050 INFO [RS:0;jenkins-hbase9:41885] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 22:55:53,050 INFO [RS:0;jenkins-hbase9:41885] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 22:55:53,050 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 22:55:53,050 INFO [RS:0;jenkins-hbase9:41885] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 22:55:53,051 INFO [RS:0;jenkins-hbase9:41885] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 22:55:53,053 INFO [RS:0;jenkins-hbase9:41885] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:41885 2023-02-09 22:55:53,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:53,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:55:53,055 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,41885,1675983241428 2023-02-09 22:55:53,055 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@58eb2727 rejected from java.util.concurrent.ThreadPoolExecutor@5cd4b8e2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,055 DEBUG [RS:1;jenkins-hbase9:44627] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/oldWALs 2023-02-09 22:55:53,055 INFO [RS:1;jenkins-hbase9:44627] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C44627%2C1675983241473:(num 1675983243019) 2023-02-09 22:55:53,055 DEBUG [RS:1;jenkins-hbase9:44627] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:53,055 INFO [RS:1;jenkins-hbase9:44627] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:55:53,056 INFO [RS:1;jenkins-hbase9:44627] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 22:55:53,056 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 22:55:53,056 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,41885,1675983241428] 2023-02-09 22:55:53,056 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,41885,1675983241428; numProcessing=2 2023-02-09 22:55:53,059 INFO [RS:1;jenkins-hbase9:44627] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:44627 2023-02-09 22:55:53,060 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase9.apache.org,41885,1675983241428 already deleted, retry=false 2023-02-09 22:55:53,060 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,41885,1675983241428 expired; onlineServers=1 2023-02-09 22:55:53,061 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,44627,1675983241473 2023-02-09 22:55:53,061 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:55:53,061 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@74062363 rejected from java.util.concurrent.ThreadPoolExecutor@7039b93b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,062 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,44627,1675983241473] 2023-02-09 22:55:53,062 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,44627,1675983241473; numProcessing=3 2023-02-09 22:55:53,064 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase9.apache.org,44627,1675983241473 already deleted, retry=false 2023-02-09 22:55:53,064 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,44627,1675983241473 expired; onlineServers=0 2023-02-09 22:55:53,064 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,44215,1675983240249' ***** 2023-02-09 22:55:53,064 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 22:55:53,065 DEBUG [M:0;jenkins-hbase9:44215] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a6b51c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-02-09 22:55:53,065 INFO [M:0;jenkins-hbase9:44215] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,44215,1675983240249 2023-02-09 22:55:53,065 INFO [M:0;jenkins-hbase9:44215] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,44215,1675983240249; all regions closed. 2023-02-09 22:55:53,065 DEBUG [M:0;jenkins-hbase9:44215] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:53,065 DEBUG [M:0;jenkins-hbase9:44215] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 22:55:53,065 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 22:55:53,065 DEBUG [M:0;jenkins-hbase9:44215] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 22:55:53,065 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1675983242633] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1675983242633,5,FailOnTimeoutGroup] 2023-02-09 22:55:53,065 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1675983242634] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1675983242634,5,FailOnTimeoutGroup] 2023-02-09 22:55:53,065 INFO [M:0;jenkins-hbase9:44215] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 22:55:53,065 INFO [M:0;jenkins-hbase9:44215] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 22:55:53,065 INFO [M:0;jenkins-hbase9:44215] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-02-09 22:55:53,065 DEBUG [M:0;jenkins-hbase9:44215] master.HMaster(1502): Stopping service threads 2023-02-09 22:55:53,065 INFO [M:0;jenkins-hbase9:44215] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 22:55:53,066 ERROR [M:0;jenkins-hbase9:44215] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-09 22:55:53,066 INFO [M:0;jenkins-hbase9:44215] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:55:53,066 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 22:55:53,067 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 22:55:53,067 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:55:53,067 DEBUG [M:0;jenkins-hbase9:44215] zookeeper.ZKUtil(398): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 22:55:53,067 WARN [M:0;jenkins-hbase9:44215] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 22:55:53,067 INFO [M:0;jenkins-hbase9:44215] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:55:53,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:55:53,067 INFO [M:0;jenkins-hbase9:44215] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 22:55:53,067 DEBUG [M:0;jenkins-hbase9:44215] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 22:55:53,067 INFO [M:0;jenkins-hbase9:44215] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:55:53,067 DEBUG [M:0;jenkins-hbase9:44215] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:55:53,067 DEBUG [M:0;jenkins-hbase9:44215] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 22:55:53,067 DEBUG [M:0;jenkins-hbase9:44215] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:55:53,067 INFO [M:0;jenkins-hbase9:44215] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.97 KB heapSize=718.99 KB 2023-02-09 22:55:53,081 INFO [M:0;jenkins-hbase9:44215] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.97 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/27d6540fb3774ab4a326b2284d5c075d 2023-02-09 22:55:53,084 INFO [M:0;jenkins-hbase9:44215] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d6540fb3774ab4a326b2284d5c075d 2023-02-09 22:55:53,084 DEBUG [M:0;jenkins-hbase9:44215] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/27d6540fb3774ab4a326b2284d5c075d as hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/27d6540fb3774ab4a326b2284d5c075d 2023-02-09 22:55:53,087 INFO [M:0;jenkins-hbase9:44215] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d6540fb3774ab4a326b2284d5c075d 2023-02-09 22:55:53,087 INFO [M:0;jenkins-hbase9:44215] regionserver.HStore(1080): Added hdfs://localhost:41659/user/jenkins/test-data/db96ad82-a3fd-6d44-da77-f162c2910976/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/27d6540fb3774ab4a326b2284d5c075d, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-09 22:55:53,087 INFO [M:0;jenkins-hbase9:44215] regionserver.HRegion(2947): Finished flush of dataSize ~607.97 KB/622561, heapSize ~718.98 KB/736232, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false 2023-02-09 22:55:53,088 INFO [M:0;jenkins-hbase9:44215] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:55:53,088 DEBUG [M:0;jenkins-hbase9:44215] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675983353067Disabling compacts and flushes for region at 1675983353067Disabling writes for close at 1675983353067Obtaining lock to block concurrent updates at 1675983353067Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675983353067Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622561, getHeapSize=736232, getOffHeapSize=0, getCellsCount=1499 at 1675983353068 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675983353068Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675983353068Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675983353072 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675983353072Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675983353085 (+13 ms)Finished flush of dataSize ~607.97 KB/622561, heapSize ~718.98 KB/736232, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false at 1675983353087 (+2 ms)Writing region close event to WAL at 1675983353088 (+1 ms)Closed at 1675983353088 2023-02-09 22:55:53,091 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 22:55:53,091 INFO [M:0;jenkins-hbase9:44215] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 22:55:53,092 INFO [M:0;jenkins-hbase9:44215] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:44215 2023-02-09 22:55:53,093 DEBUG [M:0;jenkins-hbase9:44215] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase9.apache.org,44215,1675983240249 already deleted, retry=false 2023-02-09 22:55:53,156 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:53,156 INFO [RS:0;jenkins-hbase9:41885] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,41885,1675983241428; zookeeper connection closed. 2023-02-09 22:55:53,157 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2956cdf7 rejected from java.util.concurrent.ThreadPoolExecutor@5cd4b8e2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,157 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:41885-0x1003b4b98c70001, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:53,157 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@516fabaf rejected from java.util.concurrent.ThreadPoolExecutor@5cd4b8e2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,157 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2cced7af] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2cced7af 2023-02-09 22:55:53,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:53,257 INFO [M:0;jenkins-hbase9:44215] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,44215,1675983240249; zookeeper connection closed. 2023-02-09 22:55:53,257 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@c4851ad rejected from java.util.concurrent.ThreadPoolExecutor@190aa2c2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,257 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): master:44215-0x1003b4b98c70000, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:53,257 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@d59166 rejected from java.util.concurrent.ThreadPoolExecutor@190aa2c2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,357 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:53,357 INFO [RS:1;jenkins-hbase9:44627] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,44627,1675983241473; zookeeper connection closed. 2023-02-09 22:55:53,357 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@411ce63e rejected from java.util.concurrent.ThreadPoolExecutor@7039b93b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,357 DEBUG [Listener at localhost/44743-EventThread] zookeeper.ZKWatcher(600): regionserver:44627-0x1003b4b98c70002, quorum=127.0.0.1:49544, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:55:53,357 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1e3e70d5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1e3e70d5 2023-02-09 22:55:53,357 ERROR [Listener at localhost/44743-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@39094987 rejected from java.util.concurrent.ThreadPoolExecutor@7039b93b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:55:53,358 INFO [Listener at localhost/44743] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-09 22:55:53,358 WARN [Listener at localhost/44743] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 22:55:53,361 INFO [Listener at localhost/44743] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 22:55:53,463 WARN [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid b11f84d6-5ece-443b-9b52-3e4a014154a8) service to localhost/127.0.0.1:41659 2023-02-09 22:55:53,465 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data5/current/BP-137204564-172.31.2.10-1675983237018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:55:53,466 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data6/current/BP-137204564-172.31.2.10-1675983237018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:55:53,467 WARN [Listener at localhost/44743] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 22:55:53,469 INFO [Listener at localhost/44743] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 22:55:53,575 WARN [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid 4256d769-ee79-40f3-a328-ac20eb1446de) service to localhost/127.0.0.1:41659 2023-02-09 22:55:53,578 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data3/current/BP-137204564-172.31.2.10-1675983237018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:55:53,578 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data4/current/BP-137204564-172.31.2.10-1675983237018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:55:53,579 WARN [Listener at localhost/44743] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 22:55:53,581 INFO [Listener at localhost/44743] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 22:55:53,685 WARN [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 22:55:53,685 WARN [BP-137204564-172.31.2.10-1675983237018 heartbeating to localhost/127.0.0.1:41659] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-137204564-172.31.2.10-1675983237018 (Datanode Uuid 73c0fc0c-d116-48f6-bf3b-b31855654d02) service to localhost/127.0.0.1:41659 2023-02-09 22:55:53,686 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data1/current/BP-137204564-172.31.2.10-1675983237018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:55:53,686 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/058fac7f-e167-fc06-573d-c7380c7a866a/cluster_c7358488-643f-b64d-dc86-96c9ef70be4d/dfs/data/data2/current/BP-137204564-172.31.2.10-1675983237018] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:55:53,706 INFO [Listener at localhost/44743] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 22:55:53,896 INFO [Listener at localhost/44743] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 22:55:54,123 INFO [Listener at localhost/44743] hbase.HBaseTestingUtility(1293): Minicluster is down